Org Web Adapter

Editing pages/eng_9635.org

:PROPERTIES:
:ID:       31af422b-6c81-4391-8373-3b4ea083f4b0
:ROAM_ALIASES: ENG-9635
:END:
#+TITLE: ENG-9635 - New custom fields are not created and existing are not populated after imported CSV

- source :: https://15five-dev.atlassian.net/browse/ENG-9635
- tags :: [[id:b08ac197-337b-4392-84d5-98eaead61400][15Five]] [[id:b71b60df-d439-4aa1-8c2c-265707adf8ec][Jira]] [[id:ca1d297e-764f-4c7d-90c7-93f448fee10b][People Attributes]] [[id:077e9eee-3cb9-4110-8be1-44ef4f698600][Bugs]]

* Description

#+BEGIN_QUOTE markdown
Steps to reproduce:

1. Login to the staging environment using attributes+ceo@15five.com / my15five
2. Import attached csv file
3. Check if new field created AND if old fields are populated with data from CSV

File: convertcsv (6).csv - 670 B

*ACTUAL RESULT:* Attached csv file has new fields that do not exists yet under thins account  After import , those fields are not created.

Also I've tested and confirmed that after import, existing fields are not populated wit proper data

*EXPECTED RESULT:* CSV import must correctly work.
#+END_QUOTE

* Attachments

#+BEGIN_SRC csv staging_test.csv
email,first_name,last_name,title,due_day,reviewer_email,time_zone,active_group_names,location,employee_id,job_description,strengths,saml_user_id,start_date,custom_update_dtext,custom_date_field,custom_qweqw,custom_dfgdfg,custom_newcsv
attr+ac323@15five.com,AAAAA3,LOL2,Agile Coach,Friday,attributes+se4@15five.com,America/Los_Angeles,"Departments~Development,Divisions~R&D,New Hires,New York Office,Teams~Agile Coaches",,ac,Help traincorporate teams on the agile methodology and oversee the development of agile teams to ensure effective outcomes for the organization.,"Deliberative,Individualization,Futuristic,Ideation,Includer",,,asdasd,,saladsasd,asdasd,csv text
#+END_SRC

#+BEGIN_SRC csv local_test.csv
email,first_name,last_name,title,due_day,reviewer_email,time_zone,active_group_names,location,employee_id,job_description,strengths,saml_user_id,start_date,custom_update_dtext,custom_date_field,custom_qweqw,custom_dfgdfg,custom_newcsv
attr+ac1323@15five.com,AAAAA3,LOL2,Agile Coach,Friday,paul@15five.com,America/Los_Angeles,"Departments",,ac,Help traincorporate teams on the agile methodology and oversee the development of agile teams to ensure effective outcomes for the organization.,"Deliberative,Individualization,Futuristic,Ideation,Includer",,,asdasd,,saladsasd,asdasd,csv text
#+END_SRC

The file below does a few things.

1. If there’s a field with no values, all attribute processing stops, no other
   attributes are created
2. If there’s a field with only partial values, all processing stops after that
   field. No other attributes are created nor are their values updated

I think the actual behavior is that for a given row, no more attributes are
processed once a null value is discovered
#+BEGIN_SRC csv better_test.csv
email,first_name,last_name,title,due_day,reviewer_email,time_zone,active_group_names,location,employee_id,job_description,strengths,saml_user_id,start_date,custom_donot_createme,custom_new_date_field,custom_createme,custom_allvalues
attr+ac30323@15five.com,Emma,Sweet,Engineering Coach,Friday,paul@15five.com,America/Los_Angeles,"Departments",,ac,Help traincorporate teams on the agile methodology and oversee the development of agile teams to ensure effective outcomes for the organization.,"Deliberative,Individualization,Futuristic,Ideation,Includer",,,Test,,New Value,csv text
attr+ac31323@15five.com,Colin,Powell,Agile Coach,Friday,paul@15five.com,America/Los_Angeles,"Departments",,ac,Help traincorporate teams on the agile methodology and oversee the development of agile teams to ensure effective outcomes for the organization.,"Deliberative,Individualization,Futuristic,Ideation,Includer",,,TesT,05/05/2020,New Value,csv text
attr+ac41323@15five.com,Johan,Saulpt,Milk Coach,Friday,paul@15five.com,America/Los_Angeles,"Departments",,ac,Help traincorporate teams on the agile methodology and oversee the development of agile teams to ensure effective outcomes for the organization.,"Deliberative,Individualization,Futuristic,Ideation,Includer",,,TEST,,New Value,csv text
#+END_SRC

* Notes
- Note taken on [2020-05-05 Tue 11:28]
  Realized I was adding the wrong test. The test I added initially was just the
  parser, and the parser finds the attributes in the CSV just fine. The problem
  appears in the ~ff.mods.bulkimport.utils.process_bulk_import_file~ function [fn:0]

When printing the input v. the output of the function
  ~ff.mods.bulkimport.utils.create_new_attributes~ we can see that it doesn’t
  create the correct number of attributes if the values were blank for a given attribute.

[fn:0] [[file:/usr/home/powellc/src/ff/fifteen5/ff/mods/bulkimport/utils.py::if is_people_attribute_csv_enabled:]]
- Note taken on [2020-05-05 Tue 10:29] \\
  Figured out that this happens when the CSV contains a custom attribute without
  any values. Writing tests to confirm now and will then do a fix up, should go pretty quickly.