- ID
- 82548add-c3c0-4909-912c-db0023439752
- ROAM_ALIASES
- ENG-10691
ENG-10691 - Resolve explosion of queries when bulk importing users
Description
Performance tests were add to the bulkimport API as part of ENG-10533. Next we need to comb through the function that processes the CSV file and add some serious caching, memoization or pre-fetching to avoid a ton of N+1 queries based on the number of users in the import.
The biggest offenders right now is the CompanyStruct class, which by default calls `reload_from_db` every time it's initalized, and it appears that happens 4+ times per row in the CSV file.
The other big improvement is if we just comment out this line: https://github.com/15five/fifteen5/blob/dev/ff/mods/bulkimport/utils.py#L1186
That cuts the queries down by a factor of about 3. There's a todo in the code there, but it may be possible to do the import inside the request and spin the function `create_reviewer_reporter_relationship` in a celery task? I'm not sure what the ramifications are of that though.