XParser read-in buffer too small for large-scale 0.xml files

[svdhoog]

Description of the error

Using a large-scale ABM, if there is an agent in the model that is accumulating a history of data in a dynamic array, then an error may occur if a snapshot is created after many iterations, and we try to read it in again to continue from the snapshot state. Or, a 0.xml file might also contain variables that are simply very large (more than 1,000 chars, which was the original buffer, or 100,000, which is the current buffer). This pull request introduces a global constant to set the buffer.

The error message is:

Error: agent reading buffer too small

Diagnosis

If the contents of a single memory variable (a static or dynamic array) exceeds 100 KB (100,000 chars), this exceeds the size of a read-in buffer in the file: xml.tmpl.

Cause:

This buffer is used when FLAME tries to read-in data from 0.xml file:

char arraydata[100000];

Solution 1:

In file: xml.tmpl, increase the buffer for reading in static and dynamic arrays by a factor of 10, to 1,000,000 chars, or 1 MB.

In these functions:

int read_int_static_array
int read_float_static_array
int read_double_static_array

int read_int_dynamic_array
int read_float_dynamic_array
int read_double_dynamic_array

int read_$name

increase this buffer:

char arraydata[100000];

Solution 2:

In function readAgentXML(), increase the buffer for reading in the agents and variables by a factor of 10, to 1000000 chars, or 1 MB:

int readAgentXML(char * location, ...)
{
	FILE * file;
	char c = '\0';
line 472: 	char buffer[100000]; <-- increase this buffer to [10000000];

...

line 629: if(index < 99999) index++; <-- increase this max index to 999999;
	else
	{
		printf("Error: agent reading buffer too small\n");
		printf("%s\n", buffer);
		exit(0);
	}

NOTES

• Whenever the buffer size is adjusted make sure to also check the end of the function, where there is a check on the index that it is not yet exceeded. This should be adjusted to (buffer size -1):

int readEnvironmentXML(char * location)
{
char buffer[1000000];
...
if(index < 99999) index++;
}

int readAgentXML(char * location, ... )
{
char buffer[1000000];
...
if(index < 99999) index++;
}

Solution 3:

Define a constant buffer size at the top of the file xml.tmpl:

#define CONST_BUFFER_SIZE 1000000

This is to be used for both the static and dynamic array data and the buffer:

• char arraydata[100000]

• char buffer[1000000]

char arraydata[CONST_BUFFER_SIZE];
...
char buffer[CONST_BUFFER_SIZE];
...
if(index < CONST_BUFFER_SIZE-1) index++;