NodeGame Realtime Synchronous Online Experiments In The Browser

This section centers around additional features of nodeGame (version 3.0) that have not already been covered in “Creating experiments with nodeGame”. NodeGame is under active development, so new features might be added in newer versions, however, those described here are expected to be kept also in future versions.

Authorization rules

The Web is a great source of anonymity. This is why it permits to do research on more sensitive topics or on individuals with rare conditions of interest that would not be possible to do in the lab (Mangan and Reips 2007).Footnote 11 However, synchronous experiments usually require to identify experimental participants to prevent them from entering the same game room multiple times, or to exclude those who have already played the experiment. nodeGame provides a default authorization system that stores authorization tokens in the browser. Moreover, it is possible to define a custom authorization function that can accept or reject incoming connections based on properties such as IP, browser type, etc.

nodeGame API to create authorization rules

Authorization rules are specified in the auth/ directory. File auth.settings.js allows one to disable or enable authorization checks. The most important options are:

- mode: Available modes: dummy: creates dummy ids and passwords in sequential order; auto: creates random eight-digit alphanumeric ids and passwords; local: reads the authorization codes from file; remote: fetches the authorization codes from a remote URI; custom: uses the ’customCb’ field.

- nCodes: The number of codes to create.

- addPwd: If TRUE, a password is added to each code created.

- codesLength: The length of generated codes.

- claimId: If TRUE, remote clients will be able to claim an id via GET request (useful when posting tasks on online labor markets, such as Amazon Mechanical Turk).

- claimIdValidateRequest: Authorization callback for incoming ’claimId’ requests.

Automated players: bots and phantoms

Bots and phantoms are two computer-controlled client types executed in the server (see “Client types”). The only difference between them consists of the fact that phantoms are executed in a headless browser,Footnote 12 while bots are not. A headless browser is a normal Web browser without a graphical user interface. It behaves exactly as a normal computer browser, with the only exception that rendered pages are not shown to anyone. In other words, phantoms are able to load a full HTML page like human players would do with their “headed” browser. This makes phantoms particularly suited for testing and debugging an experiment before launching it. Bots, on the other hand, cannot load HTML pages, and therefore are much more light-weighted processes. This means that an experimenter can create a large number of them without a significant memory and CPU overhead. Bots can be used to replace a disconnected human player during an experiment which requires a minimum number of participants, or to play alongside humans in an synchronous environment.

nodeGame API to handle automated players

Phantoms and bots are defined in dedicated client-type files, each one being a custom implementation of the game sequence. They are usually instantiated by a logic client type in the server-side environments, where the game developer has access to two objects of the nodeGame-server API: gameRoom and channel. They offer several methods, among which:

- gameRoom.getClientType('player'): Returns the raw player type that can be modified and adapted to behave in an automated way.

- channel.connectBot(options): Connects a bot to channel.

- channel.connectPhantom(options): Connects a phantom to channel.

Disconnections and reconnections

In the online world, participants can leave a previously joined experiment for any reason. Dropouts (disconnections) are a major problem for synchronous online experiments, as they increase both the budget and the time necessary to complete the data collection. However, dropouts not only have negative externalities. In fact, as some researchers have pointed out, the voluntary nature of online participation can often lead to the production of better-quality data than laboratory experiments, because in the laboratory subjects feel obliged to stay in the lab even when they have stopped paying attention to the task at hand (Reips 2002).

nodeGame makes available a number of solutions to handle the dropout problem. For example, it is known that dropouts can be significantly mitigated by making use of apposite measures such as ‘warm-up phases’, and ‘seriousness checks’ (Reips 2002; Reips and Krantz 2010). Those are easy to implement by just adding an extra stage in the game sequence, or by adding a dedicated game level (see “Game levels”). Alternatively, it is possible to specify a “disconnection handler” associated with a minimum number of players that need to stay connected in order for the experiment to continue. This handler can be global, i.e., throughout the whole experiment, or it can be attached to single stages or steps. In this way, the minimum-players-connected condition is verified only when really needed. In fact, it is common that some parts of an experiment can be executed with a variable number of players, while others have stricter requirements. For example, at the end of a collective behavior experiment, participants are often presented with a final questionnaire. Here, a single disconnection should not affect any other player.

nodeGame has implemented a default behavior for handling disconnections: it immediately pauses the experiment, displaying a notice to all connected clients. Simultaneously, a countdown is started, at the end of which the experimenter can decide whether to continue with less players, connect a bot player, or cancel the experimental session, redirecting the participants to an exit stage.

nodeGame API to handle disconnections/reconnections

In the logic client type, three disconnect handlers are available: minPlayers, maxPlayers, or exactPlayers. They can be a number (threshold), or an array containing up to three elements ordered as follows:

1. 1. threshold: The desired value of min/max/exact connected players.

2. 2. threshold_cb: A callback executed when the specified threshold is passed.

3. 3. recovery_cb: A callback executed when the number of connected players is correct again (viz. after the threshold_cb was executed once).

Additionally, in the logic client type, it is possible to define a reconnect property to fine-tune the reconnection parameters of the reconnecting clients.

Game levels

Game levels divide an experiment into multiple parts, each of which can have a separate waiting room and a different game sequence with distinct synchronization rules. For example, an experiment could start with a preparatory part where participants do not interact with each other, but they only answer some survey questions. Only in a second part would they reach a waiting room and form groups for synchronous play. This setup has the advantage of reducing the number of dropouts in the second part, where synchronous play is happening.

nodeGame API to create game levels

In order to add a new game level, create a new folder with the name of the level (e.g., “part2”) inside the levels/ directory. Then, inside the new game level directory, add a game/ folder, and optionally a waitroom/ folder, following exactly the same structure as for the folders with the same name at the top-level of the directory of the experiment.

Matcher

The Matcher API is currently limited to matching players into pairs. Two matching algorithms are implemented: “round robin” (also known as perfect stranger matching), and “random.” However, custom matching algorithms can be easily added into the API.

nodeGame API to match players

The following code example illustrates how to create a round robin tournament schedule for four players.

Monitor interface

The monitor interface includes the list of all games currently available, the list of game rooms dispatched, and the list client connected (see Fig. 4). The state of every client is reported, including the client type and the stage of the game they are currently in. Moreover, the interface allows one to send group or individual game commands to pause, resume, advance, or restart the game. Finally, a chat window can be opened to communicate with participants in need of assistance.

Screenshot of the monitor interface. At the top of the panel, it is possible to browse the game rooms currently active in the channel. The clients connected to the selected room are shown: two players and one logic. The middle panel allows one to send a game command to the clients selected in the upper panel; other actions are available, such as manually connecting a computer-controlled client. The lower panel permits creating and sending a custom game message to the clients selected in the top panel

Additionally, for the experimenter’s convenience, the configuration of the game is displayed in a separated tab; moreover, the content of the data/ folder (where the results are saved) is listed and available for download. The state of the server is also displayed in a separated tab.

Multiple games and access points

Self-selection bias (Kraut et al., 2004) is a known problem from which online experiments can suffer. That is, only people who are interested in a certain type of experiment participate or stay in the experiment. However, this issue can be mitigated with the use of an ad hoc countermeasure called the ‘multiple-site-access’ technique (Reips 2002; Reips and Krantz 2010). In nodeGame, the same game can have as many access points as necessary. Each access point is called an alias (see “Creating and joining a new experiment”) and can be specified in the channel configuration file: channel/channel.settings.js.

Furthermore, nodeGame not only supports running multiple access points for the same game but also supports running completely different games at the same time. Each game would then have its own access points.

Requirements

Even if authorized to take part in an experiment, a client might not actually have the technical requirements to go through all the steps. In fact, in the Internet, there are hundreds, or even thousands, of different browsers and browser versions, each one equipped with a slightly different implementation of the HTML and JavaScript standards. For example, Internet Explorer browsers below version 10 are notoriously famous for not being fully standard-compatible. They can generate glitches in the visualization of a page, or in the communication with the server. Therefore, it is necessary to test the correct functioning of nodeGame on each client before letting it into an experiment.Footnote 13

Clients connecting from mobile devices, such as tables or smartphones, also require special attention. Differences in the size of the displays should be taken into account, and clients with requirements below compliance should not be accepted. Making use of CSS frameworks helps obtaining a consistent cross-device visualization of the experiment.Footnote 14 Therefore, they should always be used when performing online experiments. Furthermore, clients connecting via mobile devices might not only experience visualization issues due to the smaller size of the display but also encounter connectivity troubles due to a sudden failure of the mobile network.

By default, in nodeGame, the following requirement checks are performed: whether JavaScript is enabled and cookies supported, if the client can load HTML pages in a dedicated iframe, and if it can communicate with the server without an excessive delay. Additional checks can be added, as seen appropriate. For example, it is possible to impose restrictions based on the location of the client. Some online labor markets allow to specify similar constraints when uploading the task, but this does not always lead to accurate results. HTML 5 Geolocation API can localize clients even at the level of single streets in dense urban areas.Footnote 15 However, clients must allow to be geolocated, and this could be introduced as a requirement for participation in the experiment.

nodeGame API to create requirement rules

Requirements rules are specified in the requirements/ directory. File requirements.settings.js allows one to disable or enable authorization checks. The most important options are:

- enabled: Boolean flag.

- maxExecTime: Limits the maximum execution time for all tests.

- speedTest: Verifies that the client can exchange a given number of messages in the specified amount of time.

- viewportSize: Verifies that the client’s screen resolution is between the min and max specified.

- browserDetect: Verifies that the client’s user agent has got the required name, type, and version (more checks available).

- cookieSupport: Verifies that the client supports cookies (‘persistent’ or ‘session’).

Templates and internationalization

nodeGame supports the dynamic creation of HTML pages from templates that are rendered upon request from a connected client. Templates allow one to write modular Web pages that can be filled with blocks of content depending on the actual configuration of the experiment. As will be explained below, the use of templates presents multiple advantages, and therefore they should be preferred over static HTML pages whenever possible.

Firstly, templates introduce a clearer separation between changeable and static parts of the user interface. For example, in an ultimatum-game-like experiment, participants would divide a certain budget B of monetary units between a bidder and a respondent. An experimenter could hard code the actual value of B in every page, but this would make the code very hard to maintain whenever the value of B is updated. Instead, by using a template, the value of B would be automatically inserted in every page by the template engine. In this way, updating the value of B in the settings of a treatment does not require further modifications in other parts of the code.

Secondly, templates can create nested layout structures that reduce the complexity of the markup of the single components and the load on the server upon requesting them. Gservers A nested layout usually consists of a fixed outer frame and a variable number of interchangeable blocks. With this configuration, instead of requesting a whole new page to the server every time, only the block actually being updated will be downloaded.

Finally, templates allow one to achieve the internationalization of an experiment, e.g., the display of the text of the experiment in different languages. This can be obtained separating the translation files into different directories (contexts) corresponding to the different prefixes of the requested languages, e.g., “en”, “de”, “it”, etc. They will be automatically matched upon receiving a new request, and the properly localized page will be rendered and sent to the client.

nodeGame API to create templates

nodeGame serves all the static files from the public/ folder. However, it is possible to dynamically create HTML pages by adding files inside the views/ folder.

The views/ folder is further divided into two sub-folders: templates/ and contexts/. The templates/ folder contains PugFootnote 16 files to create and format HTML pages. Templates support dynamic code, and allow the game developer to reuse and extend components across pages. The contexts/ folder contains JavaScript files named after the corresponding template. Each file must export a function returning the “context,” i.e., an object containing all the variables used to fill-in a template.

Timers

Response times can be used as an indicator of the type of internal reasoning process used by decision-makers. Its systematic analysis can reveal a personality trait that makes use of heuristics vs. iterative, rationale thinking (Rubinstein 2013), or whether a person is expected to play more cooperatively or more selfishly in synchronous games (Rand et al. 2012). To support research investigating these questions, nodeGame has a dedicated API for defining new timers, and measuring time intervals between specific events. Most importantly, such intervals are measured directly on the client machines, so that they network latency is excluded from calculations. game servers The precision of JavaScript timers is in the order of hundreds of milliseconds, slightly higher, but still comparable to other technologies, such as Flash, Psychtoolbox, and Java (Neath et al., 2011; Reimers and Stewart, 2015). Such differences are anyway not detectable by human subjects, and the variance in response times is approximately equivalent (Leeuw and Motz 2016).

nodeGame timers are synchronized with the flow of the game, so that they are automatically paused and resumed as the game goes. Finally, the value of a timer can be easily visualized on the screen in different formats using the appropriate widget (see “Widgets”).

nodeGame API for timer operations

Each game exposes a default timer object: node.game.timer. This timer is used to measure the time of execution of the current step and contains the default timeup function as defined by the stager. Therefore, under normal conditions, the game developer should never change its properties directly, but rather create new timers when needed using the timer API, available via the node.timer object.

Furthermore, the node.timer object allows the game developer to measure time intervals from the beginning of a step, a stage, the whole game, or a custom event. The time interval measured can be absolute or effective, i.e., the time during which a game has been paused is not counted.