I played around a bit and got it working after:
- Halving the size of the DUN_GRID_SIZE_X / Y macros
- Lowering the view size to 256x160 (I was too lazy to do the maths on what half of 384 is) - this turned out to be necessary because the camera size read from the room viewport settings is used for some calculations and having a room size smaller than this breaks the scrolling code and can cause infinite loops during generation
- Stretching the tileset sprites to half their original size and updating the tileset assets to use 16x16 instead of 32x32 as the size setting
- Stretching the door sprite to half the original size
- Adding a debug message
show_debug_message(string("render room #{2} at {0},{1} ({3} x {4})",argument2,argument3,argument4,argument5,argument6))
to the start of dungen_render_room and another one that just prints "done" to the end. (Not strictly necessary, but might help debug what's happening, especially if your changes causes infinite loops somewhere)
To convert this to isometric I think something like this would be the easiest way:
- dungen_render_tile is what places individual tiles, change it so it computes coordinates using an isometric equation instead (easiest is multiplying two diagonal vectors (the two ground-plane directions in your isometric coordinate system) with length 1 "u" and "v" with the x and y coordinates, then adding the resulting x and y components together to convert back from UV coordinates to XY coordinates) and places isotiles there (the exact details depends on how you wanna implement them but the easiest is probably layer_sprite_create)
- dungen_render_room has a segment at the end that places enemies and items in some random cells in the room, this would also need to convert the incoming XY coordinates to UV coordinates when deciding where to spawn things (you should do this when you set up the xx/yy variables only)
- dun_room_pos_free is used to check for collisions both in the generation phase and when things move around, depending on how you alter how a room is rendered (e.g. if you move away from using compatibility tiles) this will also need to be changed to check for walls etc accordingly; likewise the code that spawns doors in dungen_render_room also needs to be updated to remove walls the way they're implemented (currently hardcoded to remove compatibility tiles specifically)
Hope this helps, there's probably a bunch of things I could do to separate the generation and rendering steps better...