The purpose of the “Virtual boat to explore bimanual adaptation” project is to help understanding the adaptation of the brain to changes in its visual input.
Abstract
The purpose of the “Virtual boat to explore bimanual adaptation” project is to help understanding the adaptation of the brain to changes in its visual input. We wish to measure how fast the brain adapts to such changes, and how fast it adapts to the cancellation of these changes and the returning of the input to be as before.
The experimenter is introduced with a task that requires him to generate a routine and symmetric movement with his hands, using the model of paddling a boat in order to navigate it along a straight course in the middle of the river, with minimal deviations.
After a fixed time interval, the experimenter is introduced with an effect, both visual (boat drift) and mechanical (friction of paddles) that enforces him to change his movements, in order to keep the boat course aligned. That effect can be described using the model’s terminology as a sudden stream in the river or a shortening of one of the boat’s paddle, etc.
The effect changes in fixed time intervals, when it switches between no-effect (symmetric movements), effect on the left handle and effect on the right handle.
The following parameters are recorded:
- Time interval between the triggering of an effect and the correction applied by the experimenter (detection time), i.e. the time in which the boat was sailing in the middle of the river again
- Total deviation of the boat from the center (integration of deviation along time), which can be modeled as the distance of the boat from the middle of the river (absolute distance, of course)
- Time interval between the termination of an effect and the correction applied by the experimenter to the deviation caused by the ‘after effect’ (which is the deviation of the boat, happening after the termination of the effect. This deviation is caused by the experimenter after adapting himself to the situation in which he has to “sail” differently with his two hands, because of the effect. We assume the ‘after effect’ is present)
- Total deviation of the boat from the center because of the ‘after effect’
Test setup
Test layout
The test layout consists of the following:
- A PC running the simulation software
- Two force-feedback joysticks connected to the simulation PC
- Experimenters to be tested
- An operator to instruct experimenters before they start the simulation
- A Matlab station to analyze simulation results
Figure 1 Eye-Hand coordination diagram
Test protocol
An experimenter is sitting in front of the PC, holding both handles. The PC is running the virtual-boat simulation application, and the subject name is entered by the test operator, together with the test parameters (test duration, etc).
The experimenter is given a briefing which instructs him how to “paddle the boat” using the two handles, by rotating both handles together. The main guidelines given are:
- Rotate joysticks as equal as possible (i.e. same angle, angular velocity and radius for both), in order to keep the boat aligned and drive it in a straight line
- In case of a boat shift, return it as soon as possible to the middle of the screen, where a helper vertical line is drawn
- Be aware of changes to the friction of the handles that might cause the boat to drift, and correct it by applying more force on the stiffer handle
- Be aware of changes to the effect of the handles on the boat movements that might cause the boat to drift, and avoid it by rotating the proper handle faster
Boat movement
The algorithm of the boat movement is:
- Every full rotation of the right or the left joystick causes the boat to advance forward by a fixed distance, advance to the left or to the right (respectively) by a fixed distance and changes the direction of the boat to the left or to the right (respectively) by a fixed angle
- Every rotation that is not full causes the right / left advance and the change in the boat direction to be smaller in a ratio that depends on the radius of the rotation, i.e. the smaller the radius of the rotation is, the smaller the right / left advance and the change in the boat direction are
Test equipment
Figure 2 Test Layout diagram
Hardware
Minimal system requirements:
In order to run the simulation with no effects of slow hardware, minimal hardware requirement must be introduced:
- Pentium-III 800 MHz and above PC
- 256 Mb system RAM
- Windows XP Operating-System
- Two available USB 1.1 ports
- Minimal screen resolution: 1024×768 @ 85 Hz
Force-Feedback joysticks
Both handles used in this experiment are a USB version of Microsoft’s Sidewinder Pro Force-Feedback Joystick.
The two joysticks are from different production line which introduces minor differences between them, mostly of design variations. The base system of both joysticks is most likely to be the same, so that issue is not expected to create dissimilarities in the experiment results.
Software
Joysticks drivers
The simulation PC is connected to the two handles, which should be installed with the proper drivers. We used the native Windows drivers that are installed automatically when plugging the joysticks for the first time (Windows XP).
Display adapter drivers
In order to supply proper graphic performance, the latest drivers for the display adapter must be installed.
DirectX
The simulation software uses DirectX for 3D display and input devices interface, thus DirectX 7 and above runtime must be installed.
Visual Basic Runtime Libraries
In order to run the simulation software that was written using Microsoft’s Visual Basic, VB Runtime Libraries must be installed.
MATLAB
Simulation program dumps log file for each run, which contains information of the experimenter name, experiment date, and position of the handles and boat during experiment.
This log files can be processed and analyzed later on in a Matlab environment, using scripts.
Problems encountered
1. Problem The joysticks are not attached to the surface under them, which causes strong vibrations and possible interference to the measurements.
Solution: Attach the joysticks using some mechanical device.
2. Problem: When rotating the joysticks’ handles too far from center of rotation, the handles hit the edges, so the movement of the handle is not smooth, and clipping is shown in the results graphs.
Figure 3 Joysticks hit edges
Solution: Instructions should guide not to rotate the joysticks too far from center of rotation.
3. Problem: Center of rotations of the handles is not necessarily at the origin of the handle’s axes.
Figure 4 Joysticks spin around improper axis
Solution: Instructions should guide to attempt to rotate the handle around the actual center of rotation.
Code documentation
The project’s code was written using Microsoft’s DirectX 7.0 in Visual Studio 6.0 environment.
The main DirectX7.0 libraries were DirectInput (used for controlling the joysticks) and Direct3D (used for displaying the river and the boat and for managing the boat’s movement).
Matlab import and analysis scripts
Raw data recorded during simulation can be imported easily to Matlab environment using simple scripts. Few such scripts were written in order to process data and display it graphically by using a single function that receives the filename of the dumped data. See Appendix B: Matlab scripts for the source of those scripts.
The code contains detailed documentation inside the functions and before every function, containing their parameters and description.
See Appendix A: Project’s Code for the code and its documentation.
Figure 5 Various graphs of dumped data shown in Matlab
Graph explanation
The first two graphs show the movement of the right and left joysticks. The third graph shows the effect that was opposed on the joysticks. The sixth graph shows the advancement of the boat along the experiment. The ninth graph shows the angle of the boat along the experiment. The other four graphs show the angle and the distance from the axis center of the handles of the two joysticks.
Results
Several experiments were done on different experimenters. Here is the graphical representation of the results:
1. Barak’s results:
Figure 6 Barak’s experiment graphs
Barak did not rotate the joysticks’ handle around their real center, and he also had some short breaks while rotating the right joystick in the middle of the experiment. On the other hand, it is possible to figure the deviations of the boat because of the effects and the after-effects.
2. Daniel’s results:
Figure 7 Daniel experiment graphs
Daniel had a good experiment – he used the joysticks very well, it is easy to see the effects and the after-effects and he only had one short break while rotating the joysticks. The problems were that he had one big deviation in the middle of the experiment, when no effect was opposed on the joysticks, and that the average radius of the rotations in the right joystick was small.
3. Idan’s results:
Figure 8 Idan’s experiment graphs
On one hand, Idan did not rotate the joysticks’ handle around their real center, but the experiment was good because it is easy to see the effects and after-effects on the boat’s course deviation.
4. Nitsan’s results:
Figure 9 Nitsan’s experiment graphs
Nitsan rotated the joysticks well, except for the fact that the average radius of the rotations in the right joystick was small. The effects and after-effects on the boat’s course deviation are noticeable.
5. Ordan’s results:
Figure 10 Ordan’s experiment graphs
Unfortunately, Ordan was a good player, so the effects and after-effects had no influence on the boat’s course.
6. Shachar’s results:
Figure 11 Shachar’s experiment graphs
Shachar’s experiment is invalid, and should be excluded, because he had a serious deviation in the beginning of the experiment, even before the first effect started.
Conclusions
In most of the experiments, the effects and after-effects on the boat’s course deviation were noticeable. Not all of the experimenters had same skills with controlling the joysticks, but the results, in general, were similar.
Future research
In our opinion, the results of the experiments were not unequivocal, so we think that the definitions of the project should change in a future project / experiment.
Examples:
- The effect should start only if the boat is close to the center of the river, and it is heading straight.
- The forward advancement should be depend on the radius and speed of the rotation of the joystick, and should not be fixed.
- The player must have a motivation to rotate the joysticks as good as he can.
- The situation today is that the player can rotate the joystick in any way. A limiting force feedback effect should be imposed, such that it will force the player to rotate the joystick in an appropriate way.
- The boat model is problematic, since the boat accumulates the deviation, and not only does the player tries to adapt to the effect, he also has to consciously apply an after-effect in order to cancel the boat’s overshoot from the center of the river.
Acknowledgments
We thank Dr. Amir Karniel for managing this project; Kobi Ben-Tzvi from Rafael for guiding us all along the way; Johanan Erez for giving us a wonderful lab to work at; and Inna Karinski for solving any technical difficulty we had.