Issue
I'm having trouble identifying why my app is inconsistently running queries and js objects. Sometimes pages won't load due to failed queries, and other times the js objects hang or return inconsistent results. I suspect it might be a timeout error, but I'm not sure. The app is moderately complex and manages the marshalling of logs for export, containing around 100,000 logs delivered by 2,000 trucks from a dozen different contractors, with a focus on data integrity. It's running a self-hosted MySQL database and Appsmith 1.7.11 on a virtual machine via Docker, both hosted in the same data center. I've been programming for 40 years but hardly ever use JavaScript, so any suggestions on how to identify and fix the problem would be appreciated.
Resolution
The problem described is that an app is inconsistently running queries and JS objects, resulting in pages not loading consistently and JavaScript objects hanging or returning inconsistent results. The app is used for managing the marshalling of logs for export, containing ~100,000 logs delivered by ~2000 trucks by a dozen different contractors. It is running a self-hosted MySQL (8.0.29) database as the backend and Appsmith 1.7.11 on a virtual machine via Docker.
The solution involves using synchronous JS queries and ensuring that queries have completely finished before running the next query. The issue seemed to occur when a JS object returned with queries that hadn't completely finished, causing the next query to hang for about five minutes. The number of async queries may not be the issue.
The code example provided shows the parse_data function using await statements to run queries, followed by synchronous JS queries for processing data. The data is then inserted into the database using more asynchronous queries. The solution also includes exporting the JSON file for the team to investigate further.