Hi there! Put any of your questions down below and we will do our best to answer them! Check out our introduction videos (linked below) for more information on the jam!
Interested in learning more about BCIs? Click here!
Keen to master the P300? Click here!
Want to get a more detailed look at the BCI Game Jam and its goals? Click here!
Good question. The short answer is you can choose whichever keyboard input you prefer (space/wasd/arrows). Just make sure you document it at submission!
The longer answer is the hardware we will be using to evaluate the single-button input games allows us key-map specific thought patterns to any button on the keyboard (mostly). Since we have to train/set-up the hardware every time a game is run, it is simple enough to just use whichever input was coded by the developers. For instance, right now we have a Mario-like running game and have mapped onto "W" for jumping. If one of our participants wants to run instead (or say we use a different game), all I have to do is switch the input key (done from a GUI) or load in a different saved key-map.
Your answer has thrown me for a bit of a loop. Let me see if I understand you correctly. The Mario game you have is not a single button input game but you get it to behave like one by switching things on your end so that during the jumping parts of the game, a conscious spike of brain activity on behalf of the user causes jumping but during the running parts of the game, the same brain activity causes running? The child, then, can't play the game independently because an adult needs to switch the input mapping? To make a game that a child can play independently, I should make one that requires only one button ever? Is that all correct?
Allow me to describe a bit of the game I have in mind so you can tell if it would work. I'd like the child to have a custom experience. Suppose the first choice they make is whether their character will be a boy, a girl, a cat, a dot, or a penguin. Imagine all five choices displayed on the screen, with each option highlighted in repeating succession. So to choose the cat, the child waits until the cat is highlighted and then causes a conscious spike of brain activity. When they've made a selection, the game automatically moves to the next choice, which might be choosing a style of hat for the cat. Would that work?
Sorry for the confusing response- Yes, you are correct on multiple points! First, currently the Mario game we have cannot be played independently. We are currently helping our participant play by doing the running when the key-map is set for them to "jump" or vise versa. My example was to highlight that we can re-map to any given key for a game, e.g. changing the brain-input to be mapped to "forward" instead of "jump".
Second, as you mentioned our kids cannot play any of these games independently. Typically the games have multiple input requirements with specific timing , which is much more difficult for the participant to handle, so we typically need to step in to help. That's one of the big inspirations of this game jam! Designing games with single inputs will let them play completely independently and help give them a greater sense of independence.
Regarding your game proposal- that sounds perfect! I think our participants would enjoy that quite a bit.
As a side note (and a bit of extra information for you!), the choice/highlighting input scheme you are describing could be accomplished using a couple of different BCI input techniques.
One, as you have mentioned, would be from the user causing a specific spike in brain activity (e.g. imagining a specific movement). Alternatively, the P300 tool we will be providing is well-suited for this type of game-input. When you highlight each of the options, if you do it in a random order multiple times you will actually elicit a well-defined brain response (the P300 response) which we can identify. Each time the object the child is focused on is highlighted the P300 brain response would occur and we could then select this option for the child. We are working on some videos which (hopefully) explain the P300 better! They will be added to the main page of the jam.
Either way, throughout the jam we will be available to help you with the BCI input design aspects!
Hello, again. I know the official jam is a month away but I couldn't wait. I decided to make a game designed around the P300 response now hoping you might give me some technical feedback so that I can be certain that, when the time comes, I make a game that works. I'm sure your responses will be useful to all the jammers, not just me.
Burning Questions:
1) In my game, I present highlighted options in random order at a rate of 1 per second. Is that a good speed or would faster or slower be better?
2) In my game, the number of options available gets as high as twelve. In that case, the player must wait an average of 12 secs for the option they want to come around or come around again. What have you found to be the upper limit in the number of options / length of time between repetitions? User attention span must wane at some point.
3) There is one circumstance in my game in which the player has only two choices (in the cardinal direction walking part if they are stuck in a corner). Can the binary randomness of just two choices generate a P300 response?
4) In my game, I tried three different systems for highlighting options. Choosing an animal, there is a yellow glow, a written word label, and a spoken word. Choosing foods, there is the same glow and a wiggling animation. Choosing a cardinal direction, the arrow changes colour from grey to red. Are these all equally effective?
4) In all three cases, the currently unhighlighted options remain on screen. Would it be better if only the current option was visible?
5) I am a former kindergarten teacher and designed my game with a kindergarten-level audience in mind. Are there children in your group for whom that is a good fit?
6) There is some written language in my game but it is all conveyed verbally as well. Should jammers aim for a particular reading level?
7) I program in Delphi so I don't use Unity and I was unable to try your P300 tool. Are there important aspects I missed?
Here is a link to my first attempt: https://gophergames.itch.io/yummy-yucky . The single input in the SPACE button.
Hey there! Thanks for all the questions- I'll follow up more thoroughly on Monday when I have some time.
I completely understand the desire for having a game that works 100%, and we are hoping to release our P300 tool in the next couple weeks so people can try it out and send us technical help questions (so you are ahead of the curve!). I may try to put together a 'releasable' example as well, so developers can get an idea of what might be a good way to structure the P300 (depending on my workload though!).
I briefly looked at the prototype, and just wanted to share some quick thoughts. First- I think it looks great! I can't speak for all the centers involved regarding what our audience's ages actually are (there will be 2 other hospitals besides the one I work at who we have partnered with who have BCI participants who will be our judges), but we definitely have a range of participants. As such, I am glad to have a game that is targeted towards the younger side of the population!
There are two main concepts which jumped out to me from your prototype which could benefit in using the P300. (Also working on some general videos to release so that these ideas can be explained a bit more from a different perspective!).
I hope those help for the moment, and I'll answer all the other main questions more thoroughly soon! Glad to have your enthusiasm:).
Hi there- I wanted to dive a bit more into some of your questions. Feel free to keep asking more!
First off, I want to say I enjoy the concept of your game, and your enthusiasm for learning more about the BCI control schemes/input mechanism. Let me know if what I say doesn't make sense or needs more clarification.
Previously, when I mentioned using either the imagined thought pattern or the P300 as control schemes I wasn't sure what you were using for coding your game. The reason I had brought up the P300 is that the type of game you are proposing would work well with the P300 scheme. The problem, however, is that for the P300 to work you need to carefully time when stimulus occurs (e.g. the exact moment a highlighted option happens) so that you can look 300 milliseconds later in the recorded brain data to check if there was a response from the individual. (If so, then that was the option the child was focused on! If not, then they were probably paying attention to a different option). This timing aspect and pairing with the recorded brain signals is what we have aimed to take care of for game developers with our P300 tool in the Unity game engine. Since you are using Delphi (and I am completely ignorant to its code base/available systems etc.), I am not sure how to help implement a similar system from Delphi as my background isn't in networking or TCP/UDP at all. For that tool, we are using something called LabStreamingLayer which takes care of the network communication aspects for us. I briefly looked, but couldn't see anything from LSL which has been implemented in Delphi. (If you find one though let me know and we can try to figure it out!).
As such, you might be better off using your previous idea of sequentially highlighting the options, and we can then use the key-mapping to have a particular cognitive thought pattern map to the "Space" input. Doing so might make the selection times a bit slower and possibly more unreliable (depending on the time spent waiting on each given option), but it will definitely be able to work. With that in mind, I have tried to answer some more of your questions below, and hopefully provide some more clarity for you.
Burning Questions:
1) In my game, I present highlighted options in random order at a rate of 1 per second. Is that a good speed or would faster or slower be better?
The 1 option-per-second is fine, although you could speed up that flashing rate if you'd like (say 2 Hz - 8 Hz). There is a variety of literature out there discussing the different ways to 'optimize' the P300 response, but the "best" flash rate is variable across participants. Despite the variability, slower flash rates do tend to increase amplitude of the P300 response which means a stronger response which is easier to pick up. (Note- typical information on BCI responses are usually reported for adults, so depending the child's age and visual abilities slower flash rates can definitely be beneficial!).
2) In my game, the number of options available gets as high as twelve. In that case, the player must wait an average of 12 secs for the option they want to come around or come around again. What have you found to be the upper limit in the number of options / length of time between repetitions? User attention span must wane at some point.
Regarding options-12 options is relatively small, and definitely manageable, in terms of P300-based applications. Many typical applications of the P300 in BCI is a spelling board. This often includes at least 28 choices (alphabet + space + backspace) arranged in a matrix. Some of our participants have been able to use such boards with moderate success, so 12 options should work fine. (Also having played through your game, it seems like it should be ok! Of course we will find out for sure when we do the jam and get some of our participants to provide some feedback!).
That said, you are right in terms of attention span being something worth considering in the development aspect as we are working with children at a variety of developmental levels. Hopefully that's what your game mechanics and design will help improve!
3) There is one circumstance in my game in which the player has only two choices (in the cardinal direction walking part if they are stuck in a corner). Can the binary randomness of just two choices generate a P300 response?
Yes- binary selection is a classic way to implement the P300. All the P300 response relies on is the brain's categorization of new, unexpected events which occur when you're paying attention to something (e.g. a picture/sound/etc.). So if you have 2 different options or 10 different options the P300 can still be elicited as your brain really only cares about the 'surprise' stimulus happening to the option you're focused on.
4) In my game, I tried three different systems for highlighting options. Choosing an animal, there is a yellow glow, a written word label, and a spoken word. Choosing foods, there is the same glow and a wiggling animation. Choosing a cardinal direction, the arrow changes colour from grey to red. Are these all equally effective? 4) In all three cases, the currently unhighlighted options remain on screen. Would it be better if only the current option was visible?
This is a pretty complex topic, with a lot to unpack. The short answer is that different methods for inducing the P300 event does affect its magnitude, and thus ultimately its ability to be identified in a BCI system. It's worth noting that the traditional method for inducing P300 from visual aspects involved high-contrast changes of squares around the letters in a spelling board (e.g. from black to white, similar to your arrow-scheme). But, that doesn't mean that the other ways you've chosen to highlight options isn't helpful or effective. For your purposes, I think its definitely worth trying to increase the contrast where possible to elicit strong responses (e.g. make the glows brighter/contrasting to the background). Additionally, the use of a written word label and spoken word is (in my opinion) very helpful in the animal selection task. To me, it helps keep attention and provides plenty of clarity/fun for the game.
5) I am a former kindergarten teacher and designed my game with a kindergarten-level audience in mind. Are there children in your group for whom that is a good fit?
I think this is a good fit! Please see my other post about the appropriateness of the audience. I think its good to have a range, and think there is not nearly enough consideration given to applications for very young possible BCI users. It is an important group to include in pediatric BCI research as it goes forward. (And one I am keen on exploring!).
6) There is some written language in my game but it is all conveyed verbally as well. Should jammers aim for a particular reading level?
This is another difficult (but great!) question- We have a wide range of developmental levels in the children who use BCI. Some are non-verbal, while some have full control over speech. As such, reading levels and literacy is quite varied for our children and something I will have to get back to you on after talking to some of the OTs and clinicians more about. That said, having the verbal option available for the game is quite nice as some of the BCI users do have impaired or obscured vision in certain areas, so having the choices reinforced verbally is helpful.
7) I program in Delphi so I don't use Unity and I was unable to try your P300 tool. Are there important aspects I missed?
Please take a look at my other message to highlight some thoughts on the P300 in your game! My only other comment after playing it a couple of times is that spatial distance between options can affect the P300 (e.g. if options are too close/overlapping you might get false-positives). So you may want to spread out the animal selection options just slightly, or have them staggered in the screen!
Hope that helps, and you had a good weekend! Looking forward to your actual game submission!
I want to thank you sincerely for your thorough responses to my many many questions. I look forward to designing more games around the P300 response. I am sad to report that I will not be participating in the jam, though. I had not realized that it is the same weekend that I am performing in a musical (four shows). I look forward to playing the entries after all is sang and done and then, if you're interested, creating bespoke games for you that incorporate all of the lessons learned during the jam.
It has been my pleasure! I am always happy to discuss with others, especially with those who are so enthusiastic :).
I am sorry to hear you won't be able to participate in the jam (although you did provide an early draft of a game - so with your O.K. I'll let the kids play through it and try to get some feedback for you!). I would be happy to talk and work with you going forward in helping make BCI games. Feel free to join our discord at https://discord.gg/w3Zp3zN, as it might be an easier means for us to communicate. I wish you good luck in your performance and thanks again for your contribution!
Hi H.H. Magoo!
We are currently putting the games together and trying to make them as usable as we can for the kids during the judging portion. I was wondering if you would be able to send me either the source code for YummyYucky or could send an updated version of the game where the choices are either presented sequentially or slowed down some (at least 2 second intervals)? Let me know if you have time, as we still probably are a month out from getting all the games finished! Thanks, and I hope your performance went well!
Cheers,
Eli
Hi, I am very interested and excited in participating in this game jam. I have been researching BCI and the P300 system in Unity, but I am full of anxiety because I want to make sure the game works properly with the system. If I just make the game with a single input rather than using the P300 Unity controller, would I still need a flashing light method in order for the kids to play the game? For example, if I just made a game where you only have to jump, and made Spacebar the jumping button but did not have any flashing lights or any other stimuli, would the game still be playable?
Thanks for joining the jam! We are glad to have you:)
To answer your question- If you prefer to make a game with a single input rather than the P300 controller we have distributed that is 100% A.OK! We can use a different input method to have our children play your game without relying on the visual stimulus. So in that case we can definitely work with what you provide! (Right now we actually exploit this input method with an online knock-off version of Super Mario Brothers to play with some of our kids. We control the left and right movement, and they jump with the BCI controller).
This said, part of our draw with the P300 is in that there is a clear delay built in with the P300 that the user can see and expect. This same delay happens with the other input scheme (e.g. for single button inputs) but the delay/timing aspect is a bit less clear. On the plus side though we will be working with you (and other developers!) throughout the jam to try and help you make the best game you can and understand how these delays could be considered in different game design elements.
Hope that helped some! We are working hard to put together a Discord so that the participants who aren't in the Calgary/Edmonton area can chime in and ask us questions throughout the jam. Let me know if you have any more questions! Cheers.
That's a great question! And, like a lot of the hardware parts of BCI, one that has a lot of variables with it.
For the single-button scenario we will be primarily using the commercial Emotiv Epoch system purchased a few years ago. This system is good in some design aspects in that it lets you control aspects of the 'click output' directly (such as hold time, and latency) but only to a degree. So I can change on the system itself how long it simulates holding the button for, (e.g. 20-500 ms), but it can only take a single value per keystroke. So if you have something like classic Mario where holding down the button will give you a bigger jump, we could only get one consistent jump size usually.
Now onto your question about latency - the short answer is the latency depends on how reliably the system is picking up brain signals. A good rule of thumb (from my perspective) is you don't want to design your game such that immediate twitch reactions (<1 second window) are required. Things like needing to have a sudden change with only one second to hit the button may be hard to accomplish, as the system has to 1.) Decide you are attempting to do a specific brain pattern as compared to the other constant monitoring; 2.) Decode what that brain pattern is; 3.) Send the keystroke with all the added information to the system.
This is all then complicated by the fact that the system is constantly monitoring so that sometimes spurious brain activities might be sent as 'intended action' when they weren't actually intended. In such cases the key might be pressed immediately with no delay, or might be sent multiple times in quick succession. Back to the Mario example, this is the equivalent of trying to do a running jump, but having Mario jump much to early only for you to be locked out of the 'right' jumping window. What might be more appropriate (from my naive perspective!) are things like QTEs with more generous windows (say 2-5 seconds), or 'bars' that fill up with multiple button presses.
Finally the 'next press' of the button again depends on how well the system picks up activity. If the person is continuously providing strong signal, we can actually select the latency between sending button presses so that we can increase/decrease the number of buttons sent in succession. (This is done again through the Emotiv's software). In our current set-ups this usually is only set at 20 ms, so if a person is able to generate the right signals repeatedly (and we aren't telling the computer to hold the button) multiple inputs will be sent in quick succession.
Hope that helps some!
Thanks for the question- the truth is I am not completely sure, but honestly think it should be able to (approximately 85% sure). On the back-end of things we are using a set of libraries and functions called Lab Streaming Layer (in particular super helpful codes from LSL4Unity) to handle the data transfer to/from Unity and MATLAB. The LSL data streams use a broadcast/subscription model so they can take any name you'd want. I do know we can have multiple LSL streams produced so it would just be a matter of subscribing to the right stream and having at least 2 instances of MATLAB open to process the 2 streams. Might be able to tinker with it throughout the jam to be sure. So in summary - haven't tried it yet but love the idea!
Hello! Kendra (H.H. Magoo) here. I just noticed that my game 'Yummy Yucky' no longer appears on the submissions page for the jam. Were you able to get it working for brain interface? If so, I'd love to hear how to goes over with the children so I can make a better, more in-depth game for you in the future.
Hi Kendra!
I am not sure why it isn't showing on the submissions page for the jam...I'll double check through and see what's going on with it.
As for working with the BCI - Yes! We have gotten at least 4 kids to play through, and have their feedback recorded. We are working right now with sites in Edmonton and Toronto to get their feedback from the children who are judges at those sites as well, and I'll hopefully be able to get you some of the thoughts soon. (As well as let you know what is thought of in general!).
Anecdotally in the mean time, I can say that some of our kids enjoyed your game! In particular, they found the art and voice-over quite funny and liked that it seemed like a story. Talking with one of my colleagues in Toronto, she said she was quite partial to it as well, as she really enjoyed the 'storybook' aspect of the game. I think your instincts aiming for a younger audience was great, as it really does help it stand out as a unique experience. (Plus we all love the voice work :D ).
I'd be glad to work with you some more about how to make the game more accessible for the BCI. Right now, the biggest issue that is occurring is there was some miscommunication from my end on making selection options randomly flash . In particular, I wasn't very clear in our previous posts which has led to some confusion among various developers. There are two different BCI input methods being employed in the game jam - one which relies on random flashing, and one which does not. In your game, when the selection choices are flashing randomly it is difficult to time which selection you want due to delays in the BCI access method we are using for YummyYucky!
What could potentially help this (if you get time!) would be to switch to sequentially flashing choices (particularly in the 'Eating Food' section with the arrows). I'd be happy to discuss more with you too if you'd like to have a chat!
Hope you're doing well, and I look forward to hearing from you and updating you soon with some concrete feedback from the kids!
Cheers,
Eli
I also forgot- feel free to join our discord (Neural Matrix Entertainment) if you'd like to contact me more easily. The link is : https://discord.gg/FxerWW9 . Hope to hear from you soon!
