Seeding CSV Data into a Multi-Maestro Composition

on - last edited on

Seed data support includes the following capabilities:

Ability to access to multiple columns in the CSV file. For example, if you were to have username/password pairs, or lists of responses in a questionnaire.
Ability to allocate multiple rows per track repeat. For example, so that repeating test clips within each track can each have their own row.
Ability to control how many rows are in the “slices” sent to each server.

The following notes and restrictions apply:

The CSV file used (for example, File.csv) must be in a location accessible when the composition plays.
By default, only the first column of the CSV file is extracted, resulting in a single value for each Custom Property. Leading white space and trailing white space are trimmed from the values. See the “Advanced Procedures” section below to learn how to access values from multiple columns.
No security (such as Basic Authentication or other user ID or passwords) is supported when accessing the data via the URL.
Because tracks can repeat based upon a runtime computation, such as repeat-while, repeat-for-each, or repeat counts set by script, CloudTest doesn’t compute slices based upon the track repeat count. Instead, values are distributed evenly to each server. So, if there are 5 Maestros playing the Composition, then 1/5 of the data is distributed to each one. If a track repeats more times than the slice size, then it will wrap back to the start of the slice of data. It is up to the user to ensure that the size of the seed data is sufficient to prevent this from.

To prevent too much data from being sent around unnecessarily, the slice sizeper server is capped at 2,000 values. This maximum can be changed via the Maestro service setting SeedData.MaxServerSliceSize, which can be found in the Central > Servers > Settings tab whenever the Maestro service is selected for a given server.

Basic Example (One Column Seed Data File)

By default, only the first column of the CSV file is extracted, resulting in a single value for each Custom Property. The most basic seed data example pulls an individual value from a CSV file with one column (and a sufficient number of rows to populate all the possible values).

The steps to customize the test include the creation of:

One track custom property—the file location.
Replacement of the dynamic string value with the property path (or ISSE). For example, all occurrences of a username can be dynamically substituted from File.csv using {%% prop : track : seedData_File %%}.

For more about substituting property paths, refer to Setting Property Paths.

For more about substituting ISSEs, refer to In Situ Substitution Expressions.

Create a Custom Track Property That Points to the Seed Data File

Since the clip’s property path is based on the track custom property, it is created first. Use the following steps:

Open the test composition.
Click the Properties tab in the lower panel and then select the track from the Properties list.
Click the Custom Properties tab.
Click the green plus to create a custom property at the track level whose name starts with seedData (case sensitive) and ends with the file name. For example: seedData_File.

Set the value of the custom property to be the URL of the file that contains the seed data.
Save the composition.

When the composition is played, the data in the CSV file will be extracted and set as the value of the custom property in the track. If the track repeats, each repeat of the track will get a different value.

Substitute the Dynamic Value into the Test Clip

The Custom track Property can be referred to using a property path, ISSE, script, or anywhere else a Custom Property can be used. There can be multiple seed data properties in a single track, each with a different URL or using the same URL. Parameterization of test clips to deploy seed data can be performed manually, using Find and Replace, or using the Session Template Wizard.

Open the clip in the Clip Editor and then locate and open the message to edit by double-clicking.
Select the text value to be replaced.
With the text selected, click the Property Chooser icon.
In the Property Chooser dialog box, check “The property is not local to this message. It is specified here:”
In the Type drop-down, select Custom.
In the Starting Element drop-down, select Current Track.
Enter the name of the seed data file. For example, seedData_File.

The selected text from above is replaces as shown on the right.

Note: If there is more than one, or the number is unknown, consider using Find and Replace or the Session Template Wizard’s Variable Search and Retrieval method.

Parameters

There are some optional parameters that can be specified to control seed data behavior in certain situations. Defining additional custom properties with the name of the main Custom Property followed with --- and then by the appended parameter name can specify parameter values. All parameters are numeric values that must be greater than zero.

The available parameters are:

---columnCount
---rowsPerTrack
---maxRowsPerServer

For example, if there was a Custom Property named seedData_LoginName, then to specify the columnCount parameter for that property just specify an additional property named seedData_LoginName---columnCount.

Parameters

All parameters are numeric values that must be greater than zero.

columnCount

The columnCount parameter specifies the number of columns to retrieve from the CSV file. If the value is one, the property is set to a single value. If this parameter is not present, the value 1 is used. If the value is greater than one, the property is set to an array value.

If there are fewer columns in the CSV file than specified by columnCount, then the extra values in the array are set to null. The array will always be the size specified by columnCount.

A simple example of the use of multiple columns would be where the CSV file contains username and password pairs. In this case the CSV file might contain two columns, the first column being the username and the second being the corresponding password. Each track instance needs access to both values, so columnCount is set to 2. The Custom Property value assigned to each track instance will be an array with the two values.

rowsPerTrack

The rowsPerTrack parameter specifies the number of rows from the CSV file to distribute to each instance of each track. If the value is greater than one, then the property will be an array value, one entry in the array for each row. If this parameter is not present, the value one is used.

Note that if both columnCount and rowsPerTrack are greater than one, then the property value will be a two-dimensional array, where the first index is the row and the second index is the column.

The rowsPerTrack controls how many rows are allocated for each instance (repeat) of that Track. For example, if theclip repeats 3 times, then the value of rowsPerTrack will be (minimally) 3, since you want each instance (repeat) of that track to get 3 rows.

One example of the use of multiple rows per track would be where the track contains a serially repeatingclip and it is desired to have each repeat of theclip use a different value. The rowsPerTrack is set to a big enough value to cover the expected number of serial repeats of the Clip. Somewhere inside the Clip, a Script or ISSE would typically access the Custom Property’s array value by using theclip Repeat Index (or some other appropriate mathematical formula) as the index into the array. This logic should ideally check for, and handle, running out of values (typically by either generating an error or by wrapping back to the first value in the array).

maxRowsPerServer

The maxRowsPerServer parameter specifies the maximum “slice size” (number of rows out of the total number of rows) to distribute to each server of a Composition that plays on multiple servers.

As noted above, CloudTest doesn’t compute slices based upon the track repeat count due to runtime computations such as repeat-for-each or repeat-while.

If this parameter is not present, the value of the SeedData.MaxServerSliceSize Server Setting will be used (or 2,000 if there is no such setting present). This parameter is useful if the default value of 2,000 is not appropriate.

Some typical examples:

The track repeats more than 2,000 times per server.
The rowsPerTrack parameter is set to assign multiple rows per track instance, thus requiring the size of the slice for each server to be larger. (The slice size would typically need to be at least rowsPerTrack multiplied by the number of track repeats.)

Intermediate Example

The composition shown below includes a track that uses a Dedicated Load Server. The track’s Copy Count is 4, which will result in the track being copied to four servers. The track’s virtual users (parallel repeats) is set to 25. Since the test clip has no additional repeats set the estimated Total Virtual Users calculation shows that the composition has 100 estimated virtual users. The seed data file for this composition must have, minimally, 100 rows, if each user is to receive unique data.

The composition’s lower panel General properties also reflect these settings.

The track’s custom properties (shown by clicking the Custom Properties tab when the track is selected) also reflect the 100 virtual user count.

Note that the column count (shown right) is five. When the property path is substituted, the array values are deployed similarly to the values shown here: seedData_100[0], seedData_100[1], seedData_100[2], seedData_100[3], and seedData_100[4]. This array reflects the value of the columnCount parameter shown above.

Advanced Example

This example uses the following CSV file, which is assumed to be located at “http://hostname/SampleSeedData4.csv”:

The Composition contains a Track, named Track 1 with 3 Virtual Users that are copied to 3 servers using Dedicated Load Server and the activated Copy Count field.

The track contains the following Custom Properties:

Custom Property name	Value
seedDataSample4	http://hostname/SampleSeedData4.csv
seedDataSample4---columnCount	3
seedDataSample4---rowsPerTrack	4

The custom track properties look like this:

When the Composition plays, the following instances of the Track are given the following values for the Custom Property seedDataSample4:

Track: "Track 1 (server 1 of 3)" [0]