Technic Blog

Trending is wonky! Oh no!

Hello folks,

As I'm sure most of you have noticed, Trending is not really trending at the moment. We've noticed the issue reported by our community moderators and have addressed the issue on our end. The situation will need some time, most likely a couple days to resolve fully, for Trending to catch back up. We apologize for the inconvenience. 

We have also be aware of how easily it is for user to "remake" packs to get them to show up as Trending again. We have some ideas on how to improve the algorithm backing our Trending code to make sure it doesn't favor new packs so heavily. Existing packs do have the ability to trend again and we don't want to overshadowed. Refreshing a pack should be as simple as updating an existing pack, not spamming the system with new packs, as this this is against the Modpack Terms.

Most of all, Happy New Year Crafters! We have a lot of exciting things planned for 2017!

Sincerely,

GenPage

Technic Platform Infrastructure Blog

Hey!

As reported in our previous post, we moved a vast majority of our Platform infrastructure to DigitalOcean. We mentioned that this isn’t going to show as a direct change to anything for users, but the Platform should respond more quickly compared to how it has been. The stability of the Platform has also been improved to increase our critical system redundancies and fallover in the event of a critical issue. This means when infrastructure issues happen, you will barely notice them compared to previous incidents! Now, I'll go into more detail about what exactly we've been doing since the changes are not as visible. 

DigitalOcean Infrastructure

Moving to DigitalOcean from our previous host Linode, greatly enabled us to consider new and better solutions for the Technic Platform, the move itself was complex that required us to redo our entire code deployment setup, automated node configuration system, and rebuild our network from the ground up. This took lots of time to stage, plan, and execute with some unavoidable downtime; but we made it happen!

Database Clustering

Previously our database ran off of one massive main MySQL server. While it did its job, issues would arise if the database service needed to be brought down for maintenance or due to an underlying issue with the our providers physical node. This has been replaced with a MariaDB Galera Cluster running in a master-master replication setup. This allows us to bring down nodes at will, without impacting the availability of the data to the Platform and Launcher. Rolling security updates and optimizations are now possible on a more routine basis rather than dealing with the headache of scheduling maintenance for the entire Platform.

You can read more about Galera cluster here: http://galeracluster.com/

Error Reporting & Network Metrics

One of the biggest problems was identifying and isolating problems with the Platform. Users report problems daily ranging from the Platform to Solder to the Launcher. We didn't have a great way of cross-referencing ports from users to reproduce and isolate the problems. To help assist us with this we implemented systems to target two key things.

Error Reporting

We utilize a system called Sentry (https://sentry.io) that allows our systems to remotely report when a user experiences an error and to log it. We utilize this data to cross-references reports from Users about issues they are experiencing. We've already fixed multiple issues that were plaguing the Platform, unknown to us. These include:

  • Complete overhaul of the email backend (registerations, password resets, etc.)
  • Complete overhaul of our entire web asset system
  • Rewrote Platform  Solder integration
  • More reliable Mod sub-page listings
  • Multiple profile page tweaks
  • and numerous bug fixes...

Network Metrics

The other problem was knowing where traffic was going, monitoring the health of each individual system, and identifying bottlenecks in real-time. For this, we've setup Prometheus (https://prometheus.io/). Prometheus is an open-source monitoring and time-series database. It allows us to export custom statistics from each of our individuals core systems to reference over time. To better track/analyze this data, we setup Grafana (http://grafana.org/) metric dashboard system that plugs into the Prometheus backend. So examples of our dashboard can be seen below:

Web Performance

One big improvement you might have seen is the improvement of the Platform site load times. As mentioned earlier, we've completely revamped the way we build and deploy our static web assets (CSS, Javascript, Images, fonts, etc) and optimized caching rules to drastically improve load times. These improvements will continue throughout the site as we slowing sweep through and clean up old code.

Status Page

We have a new status page to keep you guys informed about our maintenance windows and the current status of our Platform and its components. http://status.technicpack.net

More to come!

We've made great strides over the last couple of months, but this is only the beginning. We have some cool new features that we are excited to share with you soon. We are confident in working on those features now that the Platform is in better shape. Also, I hear that a band of adventures are returning for another grand adventure! 

We've Moved to DigitalOcean

Hi.

 

 

We've moved to DigitalOcean for a vast majority of our infrastructure pertaining to the Technic Platform. While everything has been switched over we have yet to move Solder and Forums. 

While this isn’t going to show as a direct change to anything for users it should definitely keep things quick and stable compared to how they have been.

 

There are a few other changes we have planned as well that will have a more direct effect on creaters and users, so stay tuned.

 

Why The Downtime?

Hi.

 

Yesterday an admin account was compromised within the Technic Platform software we use. Our servers, databases and other assorted 'very important' things were NOT compromised, only the internal software we use to administrate the Platform itself. 

The result of this was someone used the admin account to grant a brand-new account admin access, and then went around deleting packs and changing emails to accounts. Immediately this was noticed (you can’t just delete the most popular user-created modpack of all time) and the servers themselves were shut down by me. Thus, the 503 page everyone saw for the last 12+ hours. 

 

I want to be clear that our passwords are hashed and salted and can’t be accessed from the Platform software. An admin can change passwords but can’t see what the current passwords are.

 

The result of this now is that the last day or so has been erased from the Platform’s memory and it thinks it was around 24+ hours ago. Our security has been re-done and now we offer various features. The most important one would be 2-factor authentication. It’s available for any user in your profile page if you’d like to use it (and I’d suggest you do if you run any sort of business ventures with your modpack/servers). As far as another issue happening like this it should be very unlikely as admin accounts within our software can no longer admin other admins which prevents specifically what this attack did. 

 

As for why we were attacked? There is a lot of money that is moved around because of the Platform. While we at Technic don’t run servers or accept donations, server operators use the Platform to serve their customers. Because of this there is real value that can be targeted when it comes to modpacks, servers, rankings and the accounts attached to them.  

 

Thanks for all your guys’ patience with us as we dealt with this issue. We’ll be keeping an eye out as always for anything weird that happens and hopefully have prevented other attacks that use this avenue.   

 

-KakerMix  

New Redesign

Hi.

We've gone through yet another revision of the Technic Platform and the new design is live. This new look allows more modpacks on a single page vs. the old style list, and the new design works well on *mobile* browsers which is aces. We will be going through and fixing what things we've missed and tweaking as needed.
Thanks.