Website Design: Is it good to build/rebuild static pages from dynamic content? - Developers

To preface this, this is for a website I'm making to host plugins for the Clockwork gamemode on Garry's Mod. The website would be folder based, and let's say the plugins would fall under this directory: /plugin/list/all Some examples: /plugin/list/all/some-plugin /plugin/list/all/example-do-good etc. So when building, it would create the folders like above and then get data for the plugin to use in its webpage (e.g. author, description, # of downloads, etc.). --- I'm making a website and I was wondering if it could be worth having it so that you'd build all of the webpages from data stored in a database, and then rebuild them when the data changes. I'm pretty sure this concept is already in use somewhere, but I haven't been able to find many resources on the subject. Here's what I came up with on the pros/cons: [U]Pros[/U] - Faster client reading (no need for the client to wait on the data to be pulled from the database) - Less database reading (database is only read to create/update pages instead of each time a page is opened) [U]Cons[/U] - More hard drive writing (pages are written each time a plugin is created/updated) - Longer server migration times (all pages would have to be rebuilt on a new server or transferred) - Inconsistency issues (more testing would be needed to ensure no catastrophic writing issues occur) My main goal is to make it fast for the client, I'm okay with having slower developing times if that came up. [B]Does any know if this would be a good idea and if there's any other pros/cons to it? [/B]Edit: Also the language would be PHP.

You realize that there is a fairly active web dev thread that is specifically made for these kind of questions right? [url]https://facepunch.com/showthread.php?t=1464689[/url]

[QUOTE=darth-veger;51155828]You realize that there is a fairly active web dev thread that is specifically made for these kind of questions right? [url]https://facepunch.com/showthread.php?t=1464689[/url][/QUOTE] Yes I'm aware of that thread. I figured this kind of topic may go in depth so posting it in there could lead to confusion with other conversations going alongside it.

I'm pretty sure many larger websites do this (though often into RAM only iinm), but don't quote me on this. You can build them as they're needed or as background task with prioritised queue if you're concerned about migration time. There's probably also some ready-made solution for this that you can use, but don't quote me on this. I am not a web developer.

[QUOTE=Tamschi;51155954]I'm pretty sure many larger websites do this (though often into RAM only iinm), but don't quote me on this. You can build them as they're needed or as background task with prioritised queue if you're concerned about migration time. There's probably also some ready-made solution for this that you can use, but don't quote me on this. I am not a web developer.[/QUOTE] I like the idea of using background tasks to speed up migrations, that's something I'll definitely look into adding. As for ready-made solutions, I haven't come across anything. There's probably a term for what I'm talking about but I don't know it, so finding more info on the subject is difficult.

[QUOTE=_RJ_;51155985]I like the idea of using background tasks to speed up migrations, that's something I'll definitely look into adding. As for ready-made solutions, I haven't come across anything. There's probably a term for what I'm talking about but I don't know it, so finding more info on the subject is difficult.[/QUOTE] Try "Static Cache".

It's quite common to do this ahead of time for pages that really are just static content (or whose only dynamic content is loaded externally - e.g. Disqus comments). Look up "static site generator". Alternatively, every large dynamic web application worth its salt caches [I]parts[/I] of output pages.

[QUOTE=Tamschi;51155997]Try "Static Cache".[/QUOTE] Thanks, I'll look into that as it seems promising. [QUOTE=DrTaxi;51156045]It's quite common to do this ahead of time for pages that really are just static content (or whose only dynamic content is loaded externally - e.g. Disqus comments). Look up "static site generator". Alternatively, every large dynamic web application worth its salt caches [I]parts[/I] of output pages.[/QUOTE] Hmm... Not sure if the static site generator is what I had in mind however I think a bit of a spin on it may work. I was thinking you'd have a plugin listed like so: /plugin/list/example-plugin And then in there you'd have your main page that the client access and then the page would load data from more folders inside it (probably something that just prints out the data to allow a simple API), e.g. /plugin/list/example-plugin/comments /plugin/list/example-plugin/author /plugin/list/example-plugin/version etc. That could help with client speed, that way the client wouldn't have to wait for the entire webpage to rebuild but instead just wait for whichever part they interacted with to rebuild.

What your talking about can be accomplished with lektor cms. Not sure if your married to using php, but that sounds like what your describing. Otherwise why not just make a regular dynamic website? It sounds like youre trying to pre optimize for the sake of optimization.

[QUOTE=brianosaur;51157333]Otherwise why not just make a regular dynamic website? It sounds like youre trying to pre optimize for the sake of optimization.[/QUOTE] That's also true. In all likelihood you're not gonna get anywhere near enough traffic that you'd have to start worrying about optimisation.

[QUOTE=brianosaur;51157333]What your talking about can be accomplished with lektor cms. Not sure if your married to using php, but that sounds like what your describing. Otherwise why not just make a regular dynamic website? It sounds like youre trying to pre optimize for the sake of optimization.[/QUOTE] I have a lot of past experience with PHP is my main point on using it, this is also a learning experience for myself so that's why I hope to do it the way I'm describing. As for lektor that seems pretty interesting, it reminds me of JSP for some reason (in terms of how it's written). I think I'll actually try using it in one of my next websites should the need come up. [QUOTE=DrTaxi;51157875]That's also true. In all likelihood you're not gonna get anywhere near enough traffic that you'd have to start worrying about optimisation.[/QUOTE] Yeah I don't expect it to become huge, however I do hope to move on to other audiences/markets with the same concept. What I'm doing now is more so a playground to test the waters on what I can do what need there is for what I'm making.

If this is simply a playground then you definitely shouldn't be thinking about scaling. Typically you want to build/release your project as little as possible. If a company is putting out lots of releases, typically thats a sign that code is crap and has lots of bugs (ones they keep fixing and releasing into production)

[QUOTE=brianosaur;51158606]If this is simply a playground then you definitely shouldn't be thinking about scaling. Typically you want to build/release your project as little as possible. If a company is putting out lots of releases, typically thats a sign that code is crap and has lots of bugs (ones they keep fixing and releasing into production)[/QUOTE] Oh yeah absolutely, however this is something that I wanted to try out and see if it would work. It shouldn't be too complicated, I just have to make sure I'm creating files in the write place and putting data where it's supposed to go. To make it clear I wouldn't build the website and release it each time a change is made. The user would do something (e.g. create a new plugin) and then the system would automatically build the necessary files for me. What I hoped to get out of this thread is to find out if what I'm talking about would be a bad methodology or not, and if anyone else had some insights into how to the best apply the methodology. --- One other issue I can foresee is that if I ever wanted to change the structure/format, then everything would have to be rebuilt. Or I could make it so that it'll modify necessary parts instead of recreating the entire file(s). I could see the development process being split into 2 parts. One where I change the structure at the source of the file being created (e.g. adding a new plugin), and two having a system in place to update current plugins (i.e. find/replace). That could lead me to adding a chronological backlog of steps to get to the latest structure but I don't think that would ever be needed unless I came across a really old plugin I saved somewhere that I wanted to add, so it wouldn't be worth adding any kind of history of "how-to" update structure/format (pragmatically speaking).

What you have in mind is called caching (intro [url]https://www.theukwebdesigncompany.com/articles/php-caching.php[/url]). Caching the entire page is possible but at some point it becomes very difficult to find out when to invalidate the cache for a certain page. When you cache only parts of the page (e.g. slow queries that aggregate lots of data and don't need to be realtime like number of downloads) you don't have this problem. You can use something like memcached or redis to implement this across multiple instances. But most of the times you don't need this kind of optimization. Unless you're looking at huge amounts of traffic optimize for dev speed and use a good framework. The client will still be really fast and you can start optimizing your queries and slower parts of the application once performance starts to become an issue (which it likely wont). Basically what brianosaur said :) Another thing you could look into: build a very fast API and render the page clientside (using frameworks like Angular2). That way you can put your frontend on a very fast CDN and optimize/scale your backend very easily.

[QUOTE=Kamshak;51196921]What you have in mind is called caching (intro [url]https://www.theukwebdesigncompany.com/articles/php-caching.php[/url]). Caching the entire page is possible but at some point it becomes very difficult to find out when to invalidate the cache for a certain page. When you cache only parts of the page (e.g. slow queries that aggregate lots of data and don't need to be realtime like number of downloads) you don't have this problem. You can use something like memcached or redis to implement this across multiple instances. But most of the times you don't need this kind of optimization. Unless you're looking at huge amounts of traffic optimize for dev speed and use a good framework. The client will still be really fast and you can start optimizing your queries and slower parts of the application once performance starts to become an issue (which it likely wont). Basically what brianosaur said :) Another thing you could look into: build a very fast API and render the page clientside (using frameworks like Angular2). That way you can put your frontend on a very fast CDN and optimize/scale your backend very easily.[/QUOTE] Sorry for the late reply, I didn't see your post until now. Also thanks for all the info, it's much appreciated!