将CSV转换为Json以使用Python发送给Watson-personality Insight API

时间:2018-06-25 10:40:17

标签: python json python-3.x ibm-watson personality-insights

我正在尝试转换包含某些我提取的评论的csv。我需要转换为Json作为IBM Watson个性化见解的输入。

带有评论的csv(WordFinal.csv)如下:

ID,Review,advice,con,date,employeeType,overallStar,position,pro,reviewLink,reviewNo,summary
3,ive been with amazon for 3 years before i finally resigned i started directly out of college as an area manager and promoted once a year to operation manager and was starting to get tapped for senior operations manager the progression at this company is unbelievable and growth potential is nearly limitless you get to work with a highly dedicated team that strives to raise the bar dont get me wrong you may come across individuals who also do the minimum and cut corners to get results but that is the case anywhere i was able to lead teams of up to 700 associates and indirectly oversee 1500 associates during the holiday season all at the age of 25 i was able to influence shift wide department wide building wide and even region wide procedures processes decisions for the company . my only con was the work life balance i was put on nights my entire time with amazon in order to continue to help the lower performing shifts and this would lead to having a daily schedule of wake up go straight to work usually forget to eat leave after 14 hours go to the gym eat maybe sleep wake up after 4 5 hours do it again based on location your leadership team s culture may vary drastically in california i had an extremely supportive senior and general manager team once i came to new jersey the senior team was pretty blind to their own operation super focused on creating reports continuous innovation projects in excel sheets versus just letting us go out there and do the actual improvements there was an extremely slow uptake on just do it recommendations and it was highly evident that this mentality was built into the department as well so again take it with a grain of salt ,"Cali - keep it up, you have culture nailedJersey - This is amazon, you can't raise the bar at amazon without actually being innovating and taking big steps to change things up.",my only con was the work life balance i was put on nights my entire time with amazon in order to continue to help the lower performing shifts and this would lead to having a daily schedule of wake up go straight to work usually forget to eat leave after 14 hours go to the gym eat maybe sleep wake up after 4 5 hours do it again based on location your leadership team s culture may vary drastically in california i had an extremely supportive senior and general manager team once i came to new jersey the senior team was pretty blind to their own operation super focused on creating reports continuous innovation projects in excel sheets versus just letting us go out there and do the actual improvements there was an extremely slow uptake on just do it recommendations and it was highly evident that this mentality was built into the department as well so again take it with a grain of salt ," Nov 9, 2017",Former Employee - Operations Manager,3,I worked at Amazon full-time (More than 3 years),ive been with amazon for 3 years before i finally resigned i started directly out of college as an area manager and promoted once a year to operation manager and was starting to get tapped for senior operations manager the progression at this company is unbelievable and growth potential is nearly limitless you get to work with a highly dedicated team that strives to raise the bar dont get me wrong you may come across individuals who also do the minimum and cut corners to get results but that is the case anywhere i was able to lead teams of up to 700 associates and indirectly oversee 1500 associates during the holiday season all at the age of 25 i was able to influence shift wide department wide building wide and even region wide procedures processes decisions for the company ,https://www.glassdoor.com/Reviews/Amazon-Reviews-E6036.htm/Reviews/Employee-Review-Amazon-RVW17818923.htm,empReview_17818923,Take it with a grain of salt
19, company is on an unstoppable growth trajectory amazons business model is incredible is riding a number of secular trends ecommerce cloud ai and the stock is a winner employees are making more money than expected leadership principles drive a high performance culture that focuses on customers it feels great to work on products that customers love you get to work on very difficult problems with smart people once you establish yourself as a high performer you have a high level of job security and internal mobility teams are constantly hiring and building really innovative things and you are encouraged to move around and explore teams tend to be lean and you will be asked to learn a lot quickly ownership is highly valued office environment is really desirable located in a great downtown seattle neighborhood many people walk to work bring dogs to the office and restaurants and bars are very accessible amazon veterans tend to be incredibly talented individuals and other companies realize it being successful at amazon is well respected in the industry .  work life balance can be a challenge work demands are high and teams are often too lean you have to set your own boundaries even with kind managers overachievers will feel under water frugality as a core value goes overboard if amazon doesn t have to give it you it won t no perks no free food or drinks bad coffee unsubsidized cafeterias mediocre hardware for non technical people there doesn t seem to be a morale budget and you will have few official team outings compensation policies are not employee friendly 401k matching is subpar once your signing cash bonus is fully vested your entire compensation will be base salary and stock base salary is capped at 160k across the company stock vests twice a year if you re below a director so your compensation is very lumpy stock price appreciation is taken into consideration in your total compensation targets ie if the value of previously offered shares increases the company will count that as a raise and might not grant you additional stock bonuses despite strong performance ,None, work life balance can be a challenge work demands are high and teams are often too lean you have to set your own boundaries even with kind managers overachievers will feel under water frugality as a core value goes overboard if amazon doesn t have to give it you it won t no perks no free food or drinks bad coffee unsubsidized cafeterias mediocre hardware for non technical people there doesn t seem to be a morale budget and you will have few official team outings compensation policies are not employee friendly 401k matching is subpar once your signing cash bonus is fully vested your entire compensation will be base salary and stock base salary is capped at 160k across the company stock vests twice a year if you re below a director so your compensation is very lumpy stock price appreciation is taken into consideration in your total compensation targets ie if the value of previously offered shares increases the company will count that as a raise and might not grant you additional stock bonuses despite strong performance ," Oct 29, 2016",Current Employee - Product Manager,4,I have been working at Amazon full-time (More than 3 years), company is on an unstoppable growth trajectory amazons business model is incredible is riding a number of secular trends ecommerce cloud ai and the stock is a winner employees are making more money than expected leadership principles drive a high performance culture that focuses on customers it feels great to work on products that customers love you get to work on very difficult problems with smart people once you establish yourself as a high performer you have a high level of job security and internal mobility teams are constantly hiring and building really innovative things and you are encouraged to move around and explore teams tend to be lean and you will be asked to learn a lot quickly ownership is highly valued office environment is really desirable located in a great downtown seattle neighborhood many people walk to work bring dogs to the office and restaurants and bars are very accessible amazon veterans tend to be incredibly talented individuals and other companies realize it being successful at amazon is well respected in the industry ,https://www.glassdoor.com/Reviews/Amazon-Reviews-E6036.htm/Reviews/Employee-Review-Amazon-RVW12494284.htm,empReview_12494284,Incredible growth opportunity with downsides

IBM-Watson接受的格式如下:

    {
   "contentItems": [
      {
         "content": "Wow, I liked @TheRock before, now I really SEE how special he is. The daughter story was IT for me. So great! #MasterClass",
         "contenttype": "text/plain",
         "created": 1447639154000,
         "id": "666073008692314113",
         "language": "en"
      },
      {
         "content": ".@TheRock how did you Know to listen to your gut and Not go back to football? #Masterclass",
         "contenttype": "text/plain",
         "created": 1447638226000,
         "id": "666069114889179136",
         "language": "en"
      },
      {
         "content": ".@TheRock moving back in with your parents so humbling. \" on the other side of your pain is something good if you can hold on\" #masterclass",
         "contenttype": "text/plain",
         "created": 1447638067000,
         "id": "666068446325665792",
         "language": "en"
      }   
   ]
}

我正在尝试在IBM-watson接受输出时生成输出,但是我不知道如何实现此任务。下面的代码:

import json, csv

csvfile = 'C:\\WordFinal.csv'
jsonfile = 'C:\\OutputJ.json'

fieldnames=['ID','summary']
data= {}
with open(csvfile) as cs:
    #reader = csv.DictReader(cs)
    reader = csv.DictReader(cs, fieldnames)
    for row in reader:
    csvid = row["ID"]
    data[csvid] = row

with open(jsonfile, "w",encoding='utf8') as js:
    js.write(json.dumps(data,indent=2))

和输出(我只需要csv中的ID和摘要列-第一列和最后一列):

   {
  "ID": {
    "ID": "ID",
    "summary": "Review",
    "null": [
      "advice",
      "con",
      "date",
      "employeeType",
      "overallStar",
      "position",
      "pro",
      "reviewLink",
      "reviewNo",
      "summary"
    ]
  },
  "3": {
    "ID": "3",
    "summary": "ive been with amazon for 3 years before i finally resigned i started directly out of college as an area manager and promoted once a year to operation manager and was starting to get tapped for senior operations manager the progression at this company is unbelievable and growth potential is nearly limitless you get to work with a highly dedicated team that strives to raise the bar dont get me wrong you may come across individuals who also do the minimum and cut corners to get results but that is the case anywhere i was able to lead teams of up to 700 associates and indirectly oversee 1500 associates during the holiday season all at the age of 25 i was able to influence shift wide department wide building wide and even region wide procedures processes decisions for the company . my only con was the work life balance i was put on nights my entire time with amazon in order to continue to help the lower performing shifts and this would lead to having a daily schedule of wake up go straight to work usually forget to eat leave after 14 hours go to the gym eat maybe sleep wake up after 4 5 hours do it again based on location your leadership team s culture may vary drastically in california i had an extremely supportive senior and general manager team once i came to new jersey the senior team was pretty blind to their own operation super focused on creating reports continuous innovation projects in excel sheets versus just letting us go out there and do the actual improvements there was an extremely slow uptake on just do it recommendations and it was highly evident that this mentality was built into the department as well so again take it with a grain of salt ",
    "null": [
      "Cali - keep it up, you have culture nailedJersey - This is amazon, you can't raise the bar at amazon without actually being innovating and taking big steps to change things up.",
      "my only con was the work life balance i was put on nights my entire time with amazon in order to continue to help the lower performing shifts and this would lead to having a daily schedule of wake up go straight to work usually forget to eat leave after 14 hours go to the gym eat maybe sleep wake up after 4 5 hours do it again based on location your leadership team s culture may vary drastically in california i had an extremely supportive senior and general manager team once i came to new jersey the senior team was pretty blind to their own operation super focused on creating reports continuous innovation projects in excel sheets versus just letting us go out there and do the actual improvements there was an extremely slow uptake on just do it recommendations and it was highly evident that this mentality was built into the department as well so again take it with a grain of salt ",
      " Nov 9, 2017",
      "Former Employee - Operations Manager",
      "3",
      "I worked at Amazon full-time\u00c2\u00a0(More than 3 years)",
      "ive been with amazon for 3 years before i finally resigned i started directly out of college as an area manager and promoted once a year to operation manager and was starting to get tapped for senior operations manager the progression at this company is unbelievable and growth potential is nearly limitless you get to work with a highly dedicated team that strives to raise the bar dont get me wrong you may come across individuals who also do the minimum and cut corners to get results but that is the case anywhere i was able to lead teams of up to 700 associates and indirectly oversee 1500 associates during the holiday season all at the age of 25 i was able to influence shift wide department wide building wide and even region wide procedures processes decisions for the company ",
      "https://www.glassdoor.com/Reviews/Amazon-Reviews-E6036.htm/Reviews/Employee-Review-Amazon-RVW17818923.htm",
      "empReview_17818923",
      "Take it with a grain of salt"
    ]
  },
  "19": {
    "ID": "19",
    "summary": " company is on an unstoppable growth trajectory amazons business model is incredible is riding a number of secular trends ecommerce cloud ai and the stock is a winner employees are making more money than expected leadership principles drive a high performance culture that focuses on customers it feels great to work on products that customers love you get to work on very difficult problems with smart people once you establish yourself as a high performer you have a high level of job security and internal mobility teams are constantly hiring and building really innovative things and you are encouraged to move around and explore teams tend to be lean and you will be asked to learn a lot quickly ownership is highly valued office environment is really desirable located in a great downtown seattle neighborhood many people walk to work bring dogs to the office and restaurants and bars are very accessible amazon veterans tend to be incredibly talented individuals and other companies realize it being successful at amazon is well respected in the industry .  work life balance can be a challenge work demands are high and teams are often too lean you have to set your own boundaries even with kind managers overachievers will feel under water frugality as a core value goes overboard if amazon doesn t have to give it you it won t no perks no free food or drinks bad coffee unsubsidized cafeterias mediocre hardware for non technical people there doesn t seem to be a morale budget and you will have few official team outings compensation policies are not employee friendly 401k matching is subpar once your signing cash bonus is fully vested your entire compensation will be base salary and stock base salary is capped at 160k across the company stock vests twice a year if you re below a director so your compensation is very lumpy stock price appreciation is taken into consideration in your total compensation targets ie if the value of previously offered shares increases the company will count that as a raise and might not grant you additional stock bonuses despite strong performance ",
    "null": [
      "None",
      " work life balance can be a challenge work demands are high and teams are often too lean you have to set your own boundaries even with kind managers overachievers will feel under water frugality as a core value goes overboard if amazon doesn t have to give it you it won t no perks no free food or drinks bad coffee unsubsidized cafeterias mediocre hardware for non technical people there doesn t seem to be a morale budget and you will have few official team outings compensation policies are not employee friendly 401k matching is subpar once your signing cash bonus is fully vested your entire compensation will be base salary and stock base salary is capped at 160k across the company stock vests twice a year if you re below a director so your compensation is very lumpy stock price appreciation is taken into consideration in your total compensation targets ie if the value of previously offered shares increases the company will count that as a raise and might not grant you additional stock bonuses despite strong performance ",
      " Oct 29, 2016",
      "Current Employee - Product Manager",
      "4",
      "I have been working at Amazon full-time\u00c2\u00a0(More than 3 years)",
      " company is on an unstoppable growth trajectory amazons business model is incredible is riding a number of secular trends ecommerce cloud ai and the stock is a winner employees are making more money than expected leadership principles drive a high performance culture that focuses on customers it feels great to work on products that customers love you get to work on very difficult problems with smart people once you establish yourself as a high performer you have a high level of job security and internal mobility teams are constantly hiring and building really innovative things and you are encouraged to move around and explore teams tend to be lean and you will be asked to learn a lot quickly ownership is highly valued office environment is really desirable located in a great downtown seattle neighborhood many people walk to work bring dogs to the office and restaurants and bars are very accessible amazon veterans tend to be incredibly talented individuals and other companies realize it being successful at amazon is well respected in the industry ",
      "https://www.glassdoor.com/Reviews/Amazon-Reviews-E6036.htm/Reviews/Employee-Review-Amazon-RVW12494284.htm",
      "empReview_12494284",
      "Incredible growth opportunity with downsides"
    ]
  }
}

任何想法如何获得正确的格式?

3 个答案:

答案 0 :(得分:0)

因此,我发现使用熊猫要容易得多。

首先从创建一个包含csv信息的数据框开始(假设您的文件格式正确)。

import pandas as pd
df = pd.read_csv('WordFinal.csv')

然后,您可以迭代数据框。

records = []
for index, row in df.iterrows(): 
    rec = { 
        row['ID']: { 
            'ID': row['ID'],
            'summary': row['summary']
            # ... etc
        }
    }

    records.append(rec)

您的记录对象将包含您可以保存的JSON代码。

答案 1 :(得分:0)

如果您不需要<div class="webvUI-panel__bd"> <a href="javascript:void(0);" class="webvUI-media-box webvUI-media-box_appmsg"> <% for(var i=0; i < image.length; i++) { %> <div class="webvUI-media-box__hd"> <img class="webvUI-media-box__thumb" src="<%= image[i]%>" alt=""> </div> <% } %> <div class="webvUI-media-box__bd"> <% for(var i=0; i < food.length; i++) { %> <h4 class="webvUI-media-box__title"> <%= food[i] %> </h4> <% } %> <% for(var i=0; i < price.length; i++) { %> <p class="webvUI-media-box__desc"> <%= price[i] %>$ </p> <% } %> </div> </a> </div> 以外的任何内容,只需将summary存储在summary

data

答案 2 :(得分:0)

不指定字段名称。从完整记录中提取您想要的:

with open(csvfile, "r", encoding="utf-8") as infile:
    reader = csv.DictReader(infile)

    with open(jsonfile, "w", encoding="utf-8") as outfile:
        outfile.write(json.dumps([{"ID": row["ID"], "summary": row["summary"]} for row in reader], indent=2))

礼物:

[
  {
    "ID": "3",
    "summary": "Take it with a grain of salt"
  },
  {
    "ID": "19",
    "summary": "Incredible growth opportunity with downsides"
  }
]