ENG-9635 - New custom fields are not created and existing are not populated after imported CSV

- source :: https://15five-dev.atlassian.net/browse/ENG-9635

- tags :: 15Five Jira People Attributes Bugs

Description

#+BEGIN_QUOTE markdown

Steps to reproduce:

1. Login to the staging environment using attributes+ceo@15five.com / my15five

2. Import attached csv file

3. Check if new field created AND if old fields are populated with data from CSV

File: convertcsv (6).csv - 670 B

*ACTUAL RESULT:* Attached csv file has new fields that do not exists yet under thins account After import , those fields are not created.

Also I've tested and confirmed that after import, existing fields are not populated wit proper data

*EXPECTED RESULT:* CSV import must correctly work.

#+END_QUOTE

Attachments

#+BEGIN_SRC csv staging_test.csv

email,first_name,last_name,title,due_day,reviewer_email,time_zone,active_group_names,location,employee_id,job_description,strengths,saml_user_id,start_date,custom_update_dtext,custom_date_field,custom_qweqw,custom_dfgdfg,custom_newcsv

attr+ac323@15five.com,AAAAA3,LOL2,Agile Coach,Friday,attributes+se4@15five.com,America/Los_Angeles,"Departments~Development,Divisions~R&D,New Hires,New York Office,Teams~Agile Coaches",,ac,Help traincorporate teams on the agile methodology and oversee the development of agile teams to ensure effective outcomes for the organization.,"Deliberative,Individualization,Futuristic,Ideation,Includer",,,asdasd,,saladsasd,asdasd,csv text

#+END_SRC

#+BEGIN_SRC csv local_test.csv

attr+ac1323@15five.com,AAAAA3,LOL2,Agile Coach,Friday,paul@15five.com,America/Los_Angeles,"Departments",,ac,Help traincorporate teams on the agile methodology and oversee the development of agile teams to ensure effective outcomes for the organization.,"Deliberative,Individualization,Futuristic,Ideation,Includer",,,asdasd,,saladsasd,asdasd,csv text

#+END_SRC

The file below does a few things.

1. If there’s a field with no values, all attribute processing stops, no other

attributes are created

2. If there’s a field with only partial values, all processing stops after that

field. No other attributes are created nor are their values updated

I think the actual behavior is that for a given row, no more attributes are

processed once a null value is discovered

#+BEGIN_SRC csv better_test.csv

email,first_name,last_name,title,due_day,reviewer_email,time_zone,active_group_names,location,employee_id,job_description,strengths,saml_user_id,start_date,custom_donot_createme,custom_new_date_field,custom_createme,custom_allvalues

attr+ac30323@15five.com,Emma,Sweet,Engineering Coach,Friday,paul@15five.com,America/Los_Angeles,"Departments",,ac,Help traincorporate teams on the agile methodology and oversee the development of agile teams to ensure effective outcomes for the organization.,"Deliberative,Individualization,Futuristic,Ideation,Includer",,,Test,,New Value,csv text

attr+ac31323@15five.com,Colin,Powell,Agile Coach,Friday,paul@15five.com,America/Los_Angeles,"Departments",,ac,Help traincorporate teams on the agile methodology and oversee the development of agile teams to ensure effective outcomes for the organization.,"Deliberative,Individualization,Futuristic,Ideation,Includer",,,TesT,05/05/2020,New Value,csv text

attr+ac41323@15five.com,Johan,Saulpt,Milk Coach,Friday,paul@15five.com,America/Los_Angeles,"Departments",,ac,Help traincorporate teams on the agile methodology and oversee the development of agile teams to ensure effective outcomes for the organization.,"Deliberative,Individualization,Futuristic,Ideation,Includer",,,TEST,,New Value,csv text

#+END_SRC

Notes

- Note taken on Tue May 5, 202011:28

Realized I was adding the wrong test. The test I added initially was just the

parser, and the parser finds the attributes in the CSV just fine. The problem

appears in the ~ff.mods.bulkimport.utils.process_bulk_import_file~ function [fn:0]

When printing the input v. the output of the function

~ff.mods.bulkimport.utils.create_new_attributes~ we can see that it doesn’t

create the correct number of attributes if the values were blank for a given attribute.

[fn:0] [[file:/usr/home/powellc/src/ff/fifteen5/ff/mods/bulkimport/utils.py::if is_people_attribute_csv_enabled:]]

- Note taken on Tue May 5, 202010:29 \\

Figured out that this happens when the CSV contains a custom attribute without

any values. Writing tests to confirm now and will then do a fix up, should go pretty quickly.