Post by CliffracerX in Code sharing megathread

Viewing post in Code sharing megathread

I know the feels - I'm prone to "doodling" in Unity and making little projects, just because they seem like an interesting challenge.

Senbir, and by extension, the TC-06 architecture, are LGPL-licensed. You're 100% free to redistribute mods, updates, reimplementations, etc - I believe they just have to be LGPL themselves. I'm not sure how it affects something based on the architecture (maybe I should explicitly define that as Creative Commons or something? dunno), but far as I'm concerned/can tell, you're good to go for sharing the JS version. You'd need to include a link (I think) to here, and/or the Gitlab page, but that should be the extent of extra garbage you need to do for sharing it.

I'd be interested in seeing the catch-up/timer code you've got - even if it's not perfect, it'd probably be useful in figuring out how to give Senbir better support for clockrates >180Hz. My current ideas vaguely revolve around getting some sort of average (e.g, the InvokeRepeating speed is 1f/(clockSpeed/numberOfCyclesPerInvocation), and numberOfCyclesPerInvocation=clockSpeed//180f, assuming // is a "divide and round to integer" function of some variety), but I'm not quite sure what formula would be best, and it could still cap out around some relatively low number, like 324kHz.

EDIT: *fixing derp wording because sleepy*

edorfaus6 years ago(+1)

Well, regarding reimplementations, IANAL, but I think clones (whether of games or other things) are an example of how that doesn't require following the license of the original (as long as the clone authors haven't copied any of that original's code, just written equivalent code themselves based on observed behaviour (and maybe documentation?)). (Come to think of it, isn't Mono in some ways an example of that? Having reimplemented .NET?)

So, given that I wrote it without referencing the original code, I probably don't really have to to use the same license etc. for it. On the other hand, it's not like I'm really against using the LGPL (though I'm not very familiar with it, esp. v3, as I usually tend to use simpler ones) - and I have since started looking at the code of the original (not to directly copy it but to figure out what exactly it supports/handles (actually for a different project)), so I guess that might be going into a gray area, since it might affect whatever code I write later even if I'm not trying to copy it. So I guess it would probably be safer and easier to just use the LGPL like you said.

Anyway, here you go: https://github.com/edorfaus/TC-06.js/

You can see it live here: https://edorfaus.github.io/TC-06.js/

Regarding the architecture itself, I'm not sure if that can really be protected with a copyright license, as I think the copyright can protect your documentation about that architecture, but IIUC not the actual information inside that documentation about the architecture itself? (Meaning that if someone learned it well enough to write their own docs from scratch, without copying from yours, I think they could do that without infringing.) I think you might have to get into patents to protect the information itself. But again, IANAL, and this is a pretty messy legal area IIUC. Not to mention the international aspects of it. (I'm in Norway, myself.)

However, I think the idea about using CC for that instead of GPL is probably a good one regardless, as the GPL is not really meant for documentation like that (or documents in general, really). Do look into it yourself before deciding, though, don't take my word for it. (I'm not an expert.)

That formula pair looks about right to me (maybe depending on implementation details), though you may want to tinker with that 180 number - if a lower number is more reliable (in terms of the timer actually hitting the frequency), it might be better to do that and just loop a few more times for each timer tick instead. You may also want to add some limits to avoid having someone make it try to loop 1e9 times per timer tick or something crazy like that. (Ideally it would probably adjust that relative to what the host PC can do, but that's harder to get right.)

Here's the class that handles the ticking in my implementation: https://github.com/edorfaus/TC-06.js/blob/master/clock.js

That class emits a "tick" event to run an emulator tick, while the "render-tick" event was added later for optimization purposes (the memory debugger now uses that to avoid updating its HTML elements repeatedly in one render cycle).

(I've tried to cleanly separate the various components, to keep them independent of each other (loose coupling), with just the main index.js linking them together. I do think I have a few places left with some tighter coupling that I haven't gotten rid of yet though. Feel free to ask if something's confusing, though I can't promise to always (or ever) respond quickly.)

CliffracerX6 years ago (1 edit)

I, also not a lawyer, concur with your thoughts on all the legal jargon.

Really solid JS implementation! I've spent a bit of time toying around with it, trying to write a basic Blinkenlights program - I have to admit, having an assembler has spoiled me. Before Senbir, I was toying with an older prototype of the TC-06 architecture, and made a Blinkenlights program with naught but my documentation, and manually writing out the binary code for it. I'm having a bad time of trying to do the same in the JS version, and I'm not willing to pin that on, say, unfinished debug tools or what-have-you. >.<

Classic Terminal — (Context for the surroundings: I just hijacked a previous test/doodle project that had a basic first-person controller, so it was possible to walk around and turn the computer on/off)

I think that's probably gonna be my next development project, just trying to optimize things so it's possible to run at even a meagre 1kHz. Well, that, and toying with a networking peripheral. I have an idea for the actual hardware side (i.e, what you have to work with as a programmer), but I'm not sure how to go about letting players actually connect to one another, from a UX standpoint. Do you pick a peripheral in-game, go to the debug screen, type in a real-world IP, and connect to it? Does it automatically host things, or do you have to be a host/connector? Stuff like that.

edorfaus6 years ago(+1)

Well, 1kHz would only require each 100Hz timer tick to run the actual clock cycle function 10 times (not 10k), which doesn't sound like it ought to be a problem. (And at 180Hz it'd be just 5.6 times. Lowest integer ratio would be 8 times at 125Hz.)

I'd say try it, and see how high you can make the loop count go at a given frequency before it starts to bog down, not just in theory but in actual practice. It might surprise you (in either direction), and even if it doesn't, at least then you'd have some hard data to compare with, to see if your optimization efforts are actually helping any (and how much). (Sometimes an apparently obvious optimization actually has the opposite effect.)

As for my version not having an integrated assembler, yeah, that's probably one of the reasons I haven't done much with it yet either (other than make it).

Though, on the other hand, there's still an assembler in Senbir, so I suppose we could use that to compile the program, and then copy over the result to the browser. Still nowhere near as convenient as an integrated one, but at least we wouldn't have to do the bit-twiddling manually.

I've been meaning to add save/load actually, with support for the disk image format of Senbir, so you could just load the file instead of copying the bits manually - but I haven't gotten further than including the GUI buttons for it (and then hiding them since they weren't implemented yet).

I'll probably get around to adding an assembler eventually, but it hasn't really been a high priority yet. I was primarily going for the emulator itself, and wanted to get that into decent shape first. But, I suppose having an assembler would make testing it easier, so... Maybe I should push it up the list a bit. We'll see, I guess. (What I'm currently working on is closer to the assembler than the emulator anyway... But it's not in JS, so...)

(Oh, and, in case you didn't realize, it currently comes with a default program (for testing purposes), which includes a pixel blinker.)

Regarding the networking peripheral, yeah, UX for that could get tricky. Security suggest no auto-hosting and ability to bind to specific IPs and such, but that's rather less user-friendly. I don't think I really have much advice for that, I'm more a backend dev. Implementation might get tricky too, what with (probably) needing NAT traversal and such (unless you only intend it for LAN connections or people who can set up forwarding).

By having the IP in debug, I'm guessing you've decided to make it only do predetermined point-to-point connections? (And probably just one at a time?) Otherwise it could be solved by having the program tell the peripheral which IP to connect to (which fits in a single word).

A somewhat related issue: if someone wants to test their networking code locally, before connecting to anyone else, would they have to run another instance of the game in parallel, or is it feasible to have more than one virtual computer in the same 3D environment, and connect them virtually? The answer will probably affect the UX, either way.

CliffracerX6 years ago (1 edit)

Using 125Hz as the ratio-maker works nicely - tried at 1kHz, 10kHz, 100kHz, and 1MHz. Up to 100kHz, at least, performance is well within acceptable levels for the end-user - Unity's profiler estimates the game runs at 100 frames per second at 100kHz. 1MHz takes it down to ~8-10 FPS. Mind, there's nothing fancy put in to make it more efficient - it might be possible to get better performance than that in C#.

It doesn't particularly show in the gif, I don't think, but the paint program feels MUCH smoother at 1kHz - the redraw function when moving the cursor often doesn't even end up showing because it's just running *that fast*. If 100kHz is acceptable, I'm quite confident some low-fi games should be possible with acceptable levels of performance. Running at the default-mode resolution of 16x9, a full self-modified screen redraw function would take ~576 cycles, which would let the in-game game run at ~174 FPS. The same for the Compy6c preset is a bit less optimistic - 3 FPS in 32768 cycles per full redraw. That said, most games aren't likely to need to re-draw the entire screen at once, and they might run at lower resolutions anyway.

(EDIT: To clarify, by "full self-modified cycle", I mean one where there's at least 4 operations to modify, say, a SETDATA function. Any sort of basic self-modifying code that relies on data already in the registers (like a loop's counter) tends to need 4 ops: MOVI the data being, well, modified, PMOV it or similar, use MOVO to replace it in RAM, and then actually execute it. Any sort of fancy redraw function would probably need a lot more than that.)

The Senbir file format is painfully simple, it's literally just plaintext, one address' bits on each line, with the first line being the number of addresses to load. Meant to add a "save to bytes" function of some sort that'll save it in proper binary, but...never found the right functions for it. I have seen the pixel-blinker in the default emulator state - I was just trying to recreate my old one from the original TC-06 prototype for the fun of it. ^_^

I'm thinking binding to a specific IP (and more importantly, specific port) is the way to go about things for the networking system, which means you can have multiple network ports open on your computer at once (as well as maybe an alternate computer you can spawn in that's mostly network ports, to act as a router & enable some larger networks to be built) - port-forwarding might be kind of a pain at times, but it'd be worth it for the ability to let a bunch of people connect to the same computer via different in-game network devices.

I'd like to do predetermined point-to-point connections - it makes them simple to understand, and it adds the possibility of getting to write a custom networking system (like TCP/IP) - which would be SUPER cool. Far as I'm aware, ethernet cables & whatnot are basically just big connections between computers that funnels data from one device to another - the computers' OS actually handles figuring out where the data goes. I don't think it'd be too hard to write a simple TCP/IP style networking API that can handle automatic packet forwarding and such, so it'd be possible to make a router computer (or several) that can act as LANs, or on a larger scale, potentially even a miniature internet.

An ingame IRC system is definitely among my programming end-goals - who needs Discord when you can just connect to the Senbirnet, fire up your IRC client, and connect to the official server to chat with people?

edorfaus6 years ago

Yep, not surprised you can get fairly high with C# (I figure it's generally faster than JS).

Assuming it's a linear relationship between those numbers (which it quite possibly isn't), and using 8fps (the worst) at 1MHz, you should still be able to get 60fps at 491kHz (~59fps at 500kHz). (Though that's on your system, other players might have slower ones. (Or faster.))

Some optimization might let you get at least a bit higher - like 600kHz @ 60fps at least, I'd guess, though maybe not 1MHz. Of course, it depends on how optimizable the code really is - for all I know it might already be about as fast as it will go without major effort.

Frankly, though, I think even 100kHz should be more than plenty, considering the entire idea of Senbir is for it to be a very restricted computer - so, mission already accomplished? :)

Huh, I didn't realize the first line was the number of addresses to load - I thought it was just a plain file of data words, with no header and loading simply stopping at EOF. But then I hadn't really done much more than glance at it yet...

Yeah, saving in binary can be a bit tricky, in part because the easiest APIs are generally only for text, but also because there's often more subtle things to consider regarding the format you save in. As an example, take 32-bit words like you have here, are you saving them in big-endian, little-endian or platform-endian? The latter is usually easiest (as in, what you'd do if you don't even consider it), but also makes the format less portable as there'll technically be multiple variants of it - unless you have it declare the endianness in the headers somehow, but then loading is trickier. And if you want nice extensibility, you really have to design it carefully. Text formats are often easier.

... Huh. If we consider that the file content is a program (with associated data)... Save in ELF format? :D (As if we didn't have enough complications...) ^_^'

Re: networking, if we ignore the physical and data link layers (which also have addresses, forwarding, etc.), that's more or less correct, with the wrinkle that the thing we're ignoring isn't really a point-to-point link since you can reach multiple (local) computers with it. (I could (initially did) go into quite a bit more detail about this, but it's probably not really needed.)

IIUC, I think that what you have in mind essentially means that the IP address and port you put into the peripheral settings will identify the equivalent of a physical port on a switch or network card, and the peripheral device only gives us what is essentially a bidirectional point-to-point serial link that you can send individual words (or bytes?) across, one at a time?

Then, if you want to connect more than two virtual computers together, at least one of them must have more than one network card peripheral (unless you intend to have it act differently in listen mode vs connect mode?) and be forwarding between the others (essentially implementing the function of a switch)?

That would be interesting in several ways, because it means that two-player-only cases could use the serial link directly without any switch computer or non-application networking protocol - while more complicated scenarios (like multi-application networking) would essentially require us to implement our own data link layer protocol with addresses, routing, etc. built into it - and both would be possible with the same peripheral. (Though it's more like a modem than a network card really. Or even just a serial port. (... Oooh. Implement Hayes.))

IRC could actually be in between those two - if everyone connects directly to the IRC server it doesn't require any general networking, since IRC has message routing built in. (See RFC1459 for details... no, wait, it's been updated. And come to think of it, it might be a bit too complex for the TC-06 to do quickly anyway... Unless we use a subset I suppose, that might work...)

... Man, it's been years (...decade?) since I played around with that protocol... Or even used it... I used to be on there 24/7, even wrote some bots... Still remember a server name, even, but that network shut down IIRC...

Oh, man. ^_^' I just remembered the crazy setup I used a little while for getting onto IRC... It's even kind of relevant, since it involves both a modem and a custom serial link, to use a network...

See, me and a sibling both used IRC (same channels even), but we got online with a modem and didn't really have a real LAN. I wanted to be on IRC at the same time they were, and knew a little bit of programming, so the solution we ended up with for a little while was having their PC be online (via modem), with two instances of mIRC (one for them, one for me in the background - on Win95 IIRC), with my mIRC instance scripted up to forward everything to a custom program which connected to a COM port with a null-modem serial cable link to my PC, which had another instance of that program forwarding to a mIRC instance on my PC.

So: Internet -> modem -> PC 1 -> mIRC 1 -> custom program 1 -> serial -> PC 2 -> custom program 2 -> mIRC 2 -> me.

A bit complicated, and completely ad-hoc, but it mostly worked. (Mostly. Not everything, and not perfectly, but I think it usually worked well enough for me to chat at least. (When the serial link cooperated.) No DCC obviously. Nor anything other than IRC. Since it wasn't really networking.)

(... wait... Was it a serial link or parallel? I remember messing around with the LPT port more than the COM port... Though that might have been later... eh, it's been too long. Can't really remember, and I guess it doesn't really matter much anyway. It was a homebrew job at least.)

CliffracerX6 years ago

300kHz is what I'm opting to toy with for OSdev (lets me set a nice benchmark of 5k cycles/"tick", so the OS will run at 60 FPS), and definitely fits the theme of low-fi computing. To that end, I'm working on a new sub-code for the UTL set - TIMR, which does exactly what it says on the tin. Lets you time the number of cycles that've passed since a specific TIMR call, putting it into Register 1, and means both the programs, and operating system, can try to ensure they'll always fit into that 5k timeslot so as not to disrupt the user input & whatnot. 600kHz would make for a SLIGHTLY less complicated experience (especially for, say, filling in windows - those'll take a few OS ticks to draw in, at least!), but isn't a hugely groundbreaking increase.

I assume platform-endian; it's just Convert.ToString(disk[indexToSave])+"\n" being written to the file. The thing that would make saving in binary cool is then it's like an actual disk image - it's the raw sequence of 1s and 0s that make it up, and by extension, it means it should take up exactly the amount of space on your RL harddrive as it has storage space within. Saving to ELF is certainly a neat idea, but it's not always necessarily raw executable data - if I get, say, a basic text editor working in the windowed SenbOS project, it'll be saving text files to disk that could certainly LOOK like code, but decidedly aren't.

You're pretty much spot-on with what I'm thinking - the peripheral is essentially supposed to be a literal cable that plugs into one physical network port, giving a bi-directional I/O system. You can use SETDATA to send one 32-bit packet (e.g, reading from a register or memory address, like it is with monitors) to the device on the other end, and you can use GETDATA to read any packets that device has sent you - the peripheral keeps a buffer, so you pop 'em off like key inputs. I'm trying to figure out how many ports should be the default - 2 is what I'm thinking, maybe going up to a max of 4 for the default computer you could use in Custom mode. Beyond that, I'm considering having a few alternate computers available to spawn in (as an and/or thing) for Custom mode, such as a router with a more limited peripheral set (e.g, small monitor capable of any resolution, drive, keyboard), but with a ton more network ports available. You could link a bunch of 'em together to start making something that resembles anything from the internet, to a simple LAN.

The idea of writing a custom networking system to go with this is cool - like you said, simple two-person networking doesn't need much, but you could very feasibly work on writing a TCP/IP style system that'd let you hook up a bunch of different computers. It'd probably require some cooperation between players to get it configured nicely, but could be a really neat way to share in the nerdiness of TC-06 development. Thinking about packet ideas, there should maybe be a "I AM IP <32-bit IP>" packet (sets a cached IP address for the network port it came in on), a "WHAT IS THE IP STATUS OF <4-bit port ID>" packet (IDs 1-16 are actual ports, and shares their cached IP if applicable, ID 0 is "how many ports do you have", and should be blacklistable on certain ports so you can, say, keep a LAN from being shared to the outside world that's on the last actual network port of the device), a "SET DESTINATION IP <32-bit IP>" packet used for preparing to send data, and a "SEND DATA <32 bits of data>" packet that does exactly what it says on the tin. To get around the 32-bit limitation, maybe assume all packets are sent in twos - first a "function ID" packet, and then an "arguments" packet? Oh, there also definitely needs to be a "RECEIVE DATA" packet, so it won't just think someone's calling a function on it.

The networking "BIOS" could easily route something if the requested IP was connected to it, but if it doesn't know that IP, then it would probably run through and pass the "SET DESTINATION; SEND DATA" function along to all of the device it's connected to, in the hopes one of them will successfully route it. Would definitely need to add some IP blacklisting so it won't cause a feedback loop that'd eventually end in a device trying to send that data back to its source or something like that - e.g, "BLACKLIST ME" temporarily disables the output of "WHAT IS STATUS" or similar. Yikes, things get complicated fast.

I know very little about classic IRC, just that exists, used to be a huge deal, and that it'd be a decent idea to implement an IRC system if the Senbirnet can be a thing one day, because chatting about Senbir from within Senbir is just...so cool. As I understand it, the protocol is high-quality and versatile - friend of mine said he was toying with writing a virtual CCG (think Hearthstone, Scrolls/Caller's Bane, etc) that used IRC under the hood to handle the networked aspects of the game, a while back.

My head's spinning just trying to imagine that setup! I can't really imagine what it was like before the days of universal TCP/IP, ethernet, wi-fi, and so on - I've grown up in this era, and talking about the ye-olden days of internetting with family is always pretty mind-blowing and alien. Save for the part about terrible connection speeds - those're all too familiar where I live. Not often dialup-tier terrible, but often fairly bad - if I have internet at all, that is! Depending on when you started playing Senbir, you might've seen the big warning I had up about potentially game-breaking bugs and an inability to patch them - I got the original LD-ready build uploaded about an hour before a storm blew out my local ISP for about a week. That wasn't fun.

edorfaus6 years ago

I meant the endianness wrt. a hypothetical binary format - I know of two common ways and another uncommon way to store 4-byte ints (as in, the order of those 4 bytes) that have actually been used on various platforms. Your format uses text instead of binary, so I'd assume it uses the normal way of writing numbers (most significant digit first, just using base 2 instead of 10), and endianness doesn't really come into it.

And yeah, ELF is really only appropriate if thinking of the disk image as a single program (and I'm not sure even then, really). It would probably be more appropriate to use a VM disk image format, like e.g. qcow2. But again, probably not really worth the effort at this point. (Well, I guess plain raw binary counts as a common VM image format (I use it), which shouldn't be too hard, but still.)

So, were you thinking that a single network peripheral should have multiple ports? Or just that the default computer should have multiple network peripherals, with one port each, assigned to different device numbers?

The latter would probably be easiest to use, as then you could use the protocol you suggested of simply SETDATA/GETDATA-ing to the device to write/read a word on that port, and a different network port is simply a different device number. Otherwise you'd have to design more of a protocol just for the communication between the PC and peripheral as well, making it harder to use.

Regarding custom modes, one thing I think would be neat was if that basically gave you a computer builder - you start with just the TC-06 CPU core and a slot for each device port, and for each of them you can pick which peripheral you want to be attached to that port (if any), and then configure that peripheral.

This would allow people to build a computer with whatever peripherals they need for their application - whether that is a PC with extra disks or multiple monitors, or something like a router with nothing but network devices.

It would probably require the saved-preset format to be changed, though - maybe something based on JSON so it can be structured in an extensible way (each device type could define its own config structure)?

Regarding the packets, the nice thing about all of this just being data sent across the "wire" is that players can experiment with different packets and protocols, and (if we're lucky) end up forming a consensus about which protocol to use based on what works best. That's what happened with the Internet, after all.

If you want to design a new core networking protocol, it might be an idea to look at what has already been tried to avoid some known mistakes - because yes, this gets complicated fast. Especially when you realize you can't always trust the other end of the link to not be stupid (which can be worse than them being malicious) - e.g. by using the same address as someone else (if they can set their own).

I'll note that most low-level networking works asynchronously and tends to avoid keeping state where it can (especially for things that should just work automatically or quickly), instead sending along everything that is necessary in each data packet (e.g. source address in addition to target) - and there are reasons for that. I don't know if all of those reasons apply to the Senbirnet, though.

I'll note that what you're describing with regards to packet forwarding is more like what switches do than what routers do - routers generally only forward what they're configured to forward, to the places they're configured to forward them, and anything else gets dropped. (There's often a default route, though, but that is still configuration that basically says "anything else, send to there".) Regular switches, in contrast, forward packets between their ports without any such configuration. (Managed switches may be different, but that's mostly enterprise-level stuff.)

The core routers of the Internet do have protocols they use amongst themselves to automate changing that configuration based on current conditions, but that's mostly because of the scale and conditions of the Internet, with things like multiple available paths being normal, those paths having different delay, and routers being expected to use the best path while automatically routing around problems like broken connections. (Yep - modern networking gets complicated fast even when only considering one part of it...)

Also, a common way to avoid infinite loops is to include a max hop count (sometimes called TTL (time to live)) in the packet - and then each router it passes through decrements that field, and if it's zero, drops that packet. (That's actually how traceroute is implemented, because IP routers generally send an error response back to the originator about packets that are dropped due to that.)

Regarding IRC, yes, AFAIK, it's a pretty good protocol, and quite flexible (within what it does, obviously). It's also fairly easy to understand (at least the basics), and since it's text-based (not a binary protocol), can even be used directly by humans. (I've done that with telnet, to test and try out things.)

The main problem I had with trying to use it for other things (like transferring drawing commands for a shared whiteboard) was the message limits of the server (like e.g. max 10 messages within 10 seconds), but that's because I was trying to use someone else's IRC server - if he's running his own IRC server to run the game, then he can obviously set the limits as he wants/needs to. And I suppose that kind of game doesn't really need all that many rapid-fire messages anyway.

Thinking about it, for Senbir I think the main problem with it is that it requires text parsing (and sending whole messages rather than individual characters), and that most servers have timeouts, but given a mode with decent specs, a simple client is probably doable. A full-featured server would be rather more difficult, what with having to track users and channels, route messages, etc., but we might be able to do a simplified one that works with the simple client.

It may be better (or at least easier) to just look at it for inspiration, esp. given the word-based nature of the TC-06 (while IRC is byte-based), rather than trying to implement it directly.

itch.io

Viewing post in Code sharing megathread