XML Sitemap Generator side project - the journey so far
Published : Friday 26 January 2024
One of my long-time hobbies is programming, which I used to some degree commercially when I used to perform IT work for small businesses back in the early 2000s. It was post the dotcom boom, yet many businesses and organizations had yet to make the leap in to the online world (or even embrace digital).
Early days ASP and VB
Back then, the thriving open-source community we see today was still emerging, and the tooling wasn’t up to the standards we now enjoy. Consequently, I often found myself writing programs and scripts to accomplish tasks, feeds for content syndication, and data sharing, among other things.
So, in 2005 when XML Sitemaps came along, I found myself once again crafting scripts to automate their generation. Over time, this process matured until I decided to give it a UI, which is how G-Mapper, my XML Sitemap Generator for Windows (G-Mapper), was born. Check out that sexy UI!
I submitted this to online software websites and to my surprise it became a bit of a hit and I quickly had hundreds then thousands of users. As exciting as this was, it quickly became clear that while popular the software was buggy outside how I had used it and there was a demand for more features, including enhanced editing and image sitemaps. Over a period of time (probably months) I worked away on version 2.0, still in Visual Basic 6.0
Going online with Microsoft .Net
Version 2.0 proved to be just as popular, if not more so, garnering a reasonable amount of support from the community and also generating advertising revenue - a nice motivator to sustain the project. By this time, I had been immersed in VB.Net for quite a while and started to delve into C#.
Deciding that transitioning to an online service would likely be an easier option for maintenance, but knowing my codebase wasn’t ideal, I rewrote portions of the backend, porting them over to VB.Net, and developed a .Net Forms based front. The online XML sitemap generator was born, however, it quickly proved to be enormously problematic. Code written for a computer with ample resources does not scale well to accommodate many users. My service gained more popularity than I anticipated, creating a significant problem motivating me to optimize the backend “spider engine”.
The newly enhanced VB.Net spider engine served as the foundation for both the online version and Windows Version 3.0. This upgrade allowed me to streamline code and maintenance, although a complete transition to an online service had not been realized. I had started to experiment with moving the application online, but backend requirements and the need for more advanced front end frameworks were a step too given time I had available.
Version 3.0 of the Windows Software essentially marked a complete overhaul, departing from the legacy code by integrating the new VB.net spider engine with a brand new C# front end. The newer cleaner codebase made for easier maintenance, but there was still a lot to do.
As the online and Windows products gained popularity, I eventually began accepting donations to cover my costs and it did pretty well for a while. However as time went on advertising revenue declined, seemingly due to the introduction of GDPR. I decided to implement a small annual fee for using the service. The system proved successful and turned a modest profit, making it quite rewarding as a hobby project. Over the years though I wasn’t sure what to do with it, I maintained it, but it somewhat languished as I rolled in to a new day job.
Finally after months of evenings and weekends coding, at the end of 2023 I published the new version.
I’ve thoroughly enjoyed working with SvelteKit; it’s been a breeze to get started and the community are really fantastic. That’s not to say there haven’t been some challenges and shortcomings along the way. While I have loved my experience with SvelteKit, there are certain limitations and issues I’ve encountered compared to other “frameworks”, notably express and routing. These will be the topic of a future blog post.
At this point due to time constraints I will stop supporting the Windows version, but maintain its presence for posterity. The website has had one final facelift and tidy up but my efforts will now focus on online services, with cross platform native client versions should there be a demand, but all based on a common codebase.
What’s the future?
Currently, I have some stabilizing and bug-fixing tasks on my plate. However, as any good developer knows, prioritizing rebuilding and embracing this weeks new framework is the top concern for customers …..
Furthermore, despite my enjoyment of working with SvelteKit, it does have some frustrating limitations and the recent introduction of Runes sent React vibes shivering down my spine. HTMX, the (old) new kid on the block, is something I’m eager to explore. It takes me back to the days of ASP, PHP, and MVC and would work really well with Hugo. I’ll likely delve more into this in future blog posts if I find the time in the coming weeks and months.
XML Sitemap Generator links
Thoughts for another day
There are a whole host explorations in relation to this project which I plan to cover if / when I find the time to write about them.
- Migrating to Hugo, Tailwind and Azure Static Web Apps
- My excitement and frustration with SvelteKit
- The hosting and infrastructure side of the project
- Microsoft SQL Service migration to Mongo DB
- New service management regime built around Freshdesk
- SendGrid and Microsoft Exchange online.