System Upgrades 2020 / 2021

A message from Paul Adler,
Power Diary’s CTO

I wanted to take a moment to give you a behind-the-scenes technology update at Power Diary.

You may have noticed a few brief periods of downtime recently, and for that I would like to apologise. Even though these outages have been less than 5 minutes duration, we understand they are disruptive to your business and can cause concern - especially when you don’t know the cause or the likely duration at the time. Hopefully, this message will go some way to explain what’s happening. Before we get to that though, I do want to reassure you that even in those situations, your data security is never at risk.

So why is this happening?

The short answer is that we have been upgrading our infrastructure and software. Like all technology, we need to keep up-to-date to ensure our system can keep ahead of future demands and changes in usage patterns. By that, I’m talking about more online bookings, telehealth, online forms, ever-increasing security standards, and the need to remain compatible with the various systems we integrate with and use within the system. 

Compatibility is really important. There is always the need to keep updating, otherwise, systems can fall behind and then one day when you need to install a new app or security update you can’t because the system doesn’t support it. This is the same with personal technology.

In Power Diary’s case, these upgrades are not urgent, however, we feel it is our responsibility to be proactive so we will not be met with unpleasant surprises in the future. For example, Microsoft has new web application technology called .Net Core and we’ve been progressively migrating components of our system onto this over the past 12 months. There is nothing wrong with the current system and it is still supported, but Microsoft has announced that it will not be supported in the future and that all their development focus is on the new system. We feel it’s important to embrace the new version now, and complete our migration, rather than wait until the old system starts causing problems, or limits our capacity to develop new features.

What are the upgrades?

Here are some of the specific projects we’re working on:

• Implementing a new system for in-memory storage: we’re moving to Redis which will make accessing in-memory data faster. Unfortunately this failed on two attempts under increased user load and resulted in a 3 minute outage.
• Upgrading the way our system interacts with the database: we’re moving from EF6 to Entity Framework Core. We have been working on this for 6 months and it is almost complete.
• Updating the databases from SQL Server to PostgreSQL: this improvement will offer us more flexibility to introduce new database-driven features and better performance
• Implementing new server architecture: this will allow us to better to handle fluctuating demand levels
• Upgrading the .NET platform: we’re migrating from the .NET Framework to .NET Core as Microsoft will not continue to support the existing version in the future. Most of our applications, such as the Client Portal are migrated, but the Clinic Application is not yet.
• Updating the letter and email editor: The old editor won’t support the new .NET Core platform so needed to be replaced. We implemented a new editor called Redactor 9 months ago, but the feedback wasn’t good, so we replaced it with one called Froala. This is being phased in now.

Tasks that have been completed as of today:

• Replaced our pdf generating system: This wasn’t going to be compatible with the new .Net Core platform, so we replaced it with a new service
• The client portal has been migrated to the new .Net Core system
• Entity Framework Core is 97% complete
• Upgraded the logging database to PostgreSQL
• 25% of users are on the new Froala letter and email editor

Will there be disruptions?

Whenever we embark on these types of upgrades, my philosophy is to implement them in small steps, rather than one big change. It is more difficult to design the work this way, but it is also a lot safer. Prior to releasing any changes, all work is peer-reviewed and signed off by at least two other developers, and it’s run through our automated testing and some manual testing. 

Notwithstanding this, sometimes there are scenarios related to usage patterns and load that are very difficult for us to identify in a test environment. We have systems to create high levels of user load in our test environment, but this is inevitably different from the enormous variety of activities carried out by thousands of different real users at once. There can also be interactions which vary depending on the exact data used, users’ hardware and browser configurations, the order of actions taken, so sometimes even the best testing systems cannot account for all possible combinations. 

Unfortunately, the reality is likely to be that we will have some small issues going forward while we work through these upgrades.

However, I want to reassure you that our team of developers is available around the clock, as is our support team. Whenever we release a change, our team carefully monitors for any impacts. If any issues arise, our engineers are ready to act quickly so that any downtime will be very minimal (usually less than a few minutes). Likewise, when a bug is identified, our team will work to resolve that as soon as possible (as usual, the more information you can send us about any errors help our team get to the bottom of it faster).

The irony of improving foundations 

Upgrading foundations to meet future requirements is not glamorous. It risks causing short-term disruptions and inconvenience, and the benefits may not be overtly visible for some time. 

Ironically, the easiest thing for us to do would be to avoid upgrades to Power Diary’s infrastructure until it became critical, and instead just focus on delivering a stream of new features, with bells and whistles that we know our existing and new customers would love. However, this would be akin to repainting a house, re-tiling the bathrooms and kitchen, and adding a second story, meanwhile ignoring the fact that the foundations will need to be replaced in a couple of years. It would look great superficially, but it’s short-sighted.  

This is not how we operate and never will.  

From day one we’ve taken a long-term view in everything we do as we believe it’s the only way to provide long-term quality, value, security and reliability to our customers. We know how important Power Diary is to all of the health practices that use and rely on it, and we take that responsibility very seriously. 

Whilst it’s not glamorous, the work we’re doing now is important for the long-term. It will allow us to keep providing all the features you love, as well as continuing to add new features to make your life easier, all built on the foundations you can rely on, well into the future.

Thank you for your patience as we navigate through this period. Please be assured that we know any issues are extremely frustrating, and we will do our best to avoid and minimise any incidents.

Paul Adler

Co-Founder and Chief Technology Officer
Power Diary