itch.io

https://colab.research.google.com/drive/1F5SoDy1JPvZe5lf6dnGHSkB4C6vPIFcW?usp=sh...

Duno why we didn't end up linking the colab. 

Here it is anyway in case anyone is interested. 

I might fix some things and make a pull request to add it to transformer lens later if Neel thinks that's a good idea. 

Maybe modifying the original neuroscope so it has a selector to change between showing layers and showing an specific Neurons would be better idea though . 

Also about speed, it's not that bad. 

It still takes a moment to load in colab wich can be anoying but it was still usable. (And for whatever reason it was near instant when Alex ran the notebook locally) 

That said we were trying whith gpt small maybe it's worse whith bigger models. 

I want to note that we weren't the first to come up with "let's think step by step",https://arxiv.org/abs/2205.11916 this paper might be the origin but I wouldn't be surprised if a lot of people discovered it independently before, it's not that hard to notice if you play around with models.
It's not special either, other prompts that mean the same also seem to work.
What's novel is that as far as I know nobody had tried to use it to reduce biases or  in the inverse scaling price datasets.
(And in general the effects of using let's think step by step style prompts seems pretty unexplored so far in general).

(Note:In case it's not obvious, you can send one of your characters to the floating islands around by clicking on them after selecting the character)