API Design for JSON auto Loader Advices

Hello.
Currently i trying to make something like library for working with big amounts of data-like content for S&box, and i am not so sure about how it should be organized first of all for convenience.
data-like contents it is something like hundreds or thousands of JSON files that describe some generic objects in games (rpg, surivival or rouglikes as an example.). JSON files can be really convenient for such stuff, so i think it can be really cool stuff, and using generic entity templates to make big amount of parametrizable items, enemies and other content very fast. Look at how content organized in Cataclysm DDA or DwarfFortress (but DF uses its own data format)

How i see it currently:
You can register in your Game class something like Loader. that based on BaseFileSystem, and you can regster any C# Classes, that can be deserialized automatically when game loads. At any time you can register JSONData, specify folder when your json files should be suited, and it loads it recursively.

Example type:
Screenshot_1232
Example JSON representation:
Screenshot_3
Current API:
Screenshot_4

I Know that some stuff are really bulky, but i want to ask expierenced devs about what kind of methods can be added for stuff like that?

My additional ideas:

  • Fully change reloading to FileWatcher. But IDK about perfomance hits, because you need to filewatch for every Template Type, but there can be a lot of them.
  • Not only loading but saving JSON. Technically it can be implemented. You can load and save JSON data easily, use it something like save system in some way.
1 Like

I can’t help but ask, why not just use a database? You can serialize and deserialize your data into a database the exact same way as using JSON Files, but without the headache of managing/maintaining/backing up all these loose files.

1 Like

This is an intresting idea, but i think use database mostly for static files can be really verbose. I also have some concerns about using MongoDB (or another json-like db):

  • Accessing to DB big amount of data has its own perfomance drawback (not sure that is worse that using static files)
  • File structure can be more convenient for easy editing
  • MongoDB has its own (and it can very big sometimes) memory and runtime cost. Most of the features from MongoDB are not be used. VS just several classes that works only when you need to reload (or write) data.I always think that there is a lot to spend “free perfomance”
  • It should easily work for local lobby sessions, not only dedicated servers.

But honestly, i think its a intresting idea theoretically, i think i could try replace with DB if my concerns are wrong.

1 Like

I don’t quite understand, what problem are you trying to solve? Who would create these objects and how they will be used?

If this is a general idea on how to make a simple human-editable interface then consider using file formats designed for this, like YAML. JSON isn’t great for making files by hand, you have to use with scopes, quotes every time and this can be frustrating when dealing with hundreds of files.

1 Like

This objects will be created by developers of game (currently just me), that uses big amount of content, with some generic components and parameters. Again, check data in Cataclysm DDA, how it solves that problem. It is not an ideal, so i trying to solve that problem by myself.

I already knew about YAML, its really good stuff, but it inferiors to JSON in speed by a ten, or even more times (link), ecpecially when amount of files increases rapidly (Check some heavily modded game like Rimworld or Cataclysm DDA again).
I Dont think JSON files are too hard to manage and create. At least there is some JSON visual editors.
But if i was wrong about YAML perfomance, is there already some fast enough and usable for s&box lib for YAML, i can start to use it instead JSON.

A trade-off in usability and performance can be the use of a YAML to JSON converter.

1 Like

Yo would you be free to contact me via discord so we can chat further about this? My discod is Kaptian Core#0001

2 Likes

I made a friend request

1 Like