The good news is that the PagerDuty Operations Cloud can support your Crisis Response Management Operations for those next level threats. We know this because it’s what we use to manage the unpredictability of the global operating environment and to keep our employees safe.

In this blog, we cover the evolving nature of global preparedness and the convergence of threats along with how you can enhance your crisis response management operations with PagerDuty.

We’re not in Kansas Anymore

PagerDuty runs an annual Global Preparedness campaign to provide its employees with the tools and resources they need to be resilient in a rapidly changing world. We want our employees globally to establish best in class personal and professional resilience so we can do our very best work in support of our customers. We do this by enabling our employees to act in moments of crisis. Throughout September, we deliver emergency response guides, CPR-certification classes, emerging threat training sessions and disaster response movie trivia. Our customary volunteering opportunity in partnership with our Social Impact team and our crisis response system tests followed by a functional drill are mainstays in our annual programming.

Business leaders should always plan for the company’s worst day before it becomes a reality and because we’re anticipating that future crises will require more complex responses and tools to resolve, we’re also highlighting perfect storm scenarios across categories of threats during this year’s campaign. For example, the intensity of cyber threats and extreme natural disasters that we’re seeing over the last year or so creates an opportunity to examine the effectiveness of our crisis response management operations against combined threats. 

One thing is certain when keeping in mind Murphy’s Law: if a crisis can happen, it will happen. However, it may not happen the traditional way you planned and prepared for it. In other words, buckle up because Kansas is no more.

PagerDuty for PagerDuty

PagerDuty’s Crisis Response Management Operations Guide was built to showcase how PagerDuty uses PagerDuty. In regards to preparedness, we leverage our platform for periodic functional exercises and crisis simulations to ensure our teams are ready as we scan the horizon for emerging threats. After these exercises or simulations, we complete our postmortems (i.e., after action report or hotwash) in the platform capitalizing on all of the “Incident Action Plan” information both we and the system captured/generated during the simulated response, i.e., timeline of events, responder requests, hand-off times, status updates, notes.

For operational resilience, when we do need to respond to a critical event, the operations side just works. Using PagerDuty, we don’t have to worry about contact information being out of date, which conference bridge we should be using and where the latest version of our playbook is stored. These problems slow mean time to respond and ultimately the speed at which a critical event can be resolved. PagerDuty supports 700+ native integrations and even more with our API so we never miss a notification or need to spend time punching in conference call numbers on a dialpad. With the ability to add documentation links into our services, our plans and playbooks also never get lost.

At PagerDuty, we’re empowering teams to build the future and using our own platform to respond to the increasingly complex crises that threaten our people and operations. If you want to learn how PagerDuty can help you modernize your crisis response management operations, read our Crisis Response Management Ops Guide and sign up for a free trial.

The post Building Trust with our Customers with PagerDuty for PagerDuty: Crisis Response Management Operations appeared first on PagerDuty.

While these numbers are grim, innovation in healthcare and financial technology is opening the door to opportunities for people to gain access to healthcare and financial assistance on a global scale. Enter mTOMADY, which got its start as a project of the NGO Doctors for Madagascar, and is a digital platform that facilitates resilience to medical impoverishment and universal health coverage, particularly in countries with underserved healthcare systems such as Madagascar.

And as a PagerDuty Impact Pricing customer, mTOMADY is one of the organizations committed to helping bring essential medical care to people who need it, in the hardest-to-reach places, in the moments they need it most.

Increasing Healthcare Access Through Technology

Samuel Knauss, a medical doctor and digital health clinician, oversees mTOMADY’s Product team and is responsible for the organization’s development and operations of end-user software solutions, as well as strategy around the distribution of care and support for their patients and users through those solutions. “Our teams are distributed across Germany, Madagascar, and parts of Sub-Saharan Africa,” shared Knauss. With a small, distributed team operating in different workstreams, it’s important that communications are properly directed to the right people at the right time. “Between the team of on-the-ground agents and software developers, it is important for our teams to be able to communicate timely and effectively when incidents with our systems occur.”

mTOMADY’s services have to be online and available 24×7 in extremely rural areas to serve patients, so uptime and connectivity is critical to ensuring patients have access to proper care and financial support solutions, especially during global crises.

mTOMADY is built upon the foundation of five main principles:

• Transparency. Collecting real-time data on healthcare system usage, disease prevalence, and treatment costs enables impact to be traced down to individual health outcomes.
• Universal Access. Technology that works in resource poor settings, on any mobile phone, allows patients and healthcare workers to access the platform even in isolated, rural areas.
• Cost Reduction. Simple-to-use technology that can be easily integrated into existing procedures to reduce costs and streamline administration obstacles, improving operational efficiency.
• Advanced Technology. Cutting-edge pattern recognition and data analytics provide actionable insights to prevent insurance fraud and improve quality of care.
• Scientific Evidence. Research conducted to collect measurable, empirical evidence on how the solution works in the real world enables continuous optimization.

Manual vs. Real-Time Incident Management

Before PagerDuty, mTOMADY’s incident management process was completely manual. They used a spreadsheet for on-call directories, which led to delays in response and confusion around how to prioritize alerts for certain teams with different expertise. The manual directory also created a bottleneck in terms of scheduling because on-call rotations were globally distributed and cascaded across time zones. “Organizing our duty times and who’s responsible for what at what time is crucial in our field. If we don’t know who’s on call, we can’t provide the level of care we promise to our patients,” Knauss explained.

Since the implementation of PagerDuty, mTOMADY has seen significant improvement in their on-call and incident management processes. They replaced manual on-call spreadsheets with automated rotation schedules, which let responders know precisely when they are on call and what services they are responsible for. They also leverage PagerDuty’s ecosystem of over 350 integrations to centralize tools like Kibana, Sentry, and Slack into a single point of ingestion. With these implementations, the mTOMADY team has seen significant improvements in metrics like MTTA/MTTR, as well as better alignment around who the point person is for incidents involving certain applications and services.

“Timeliness is the most crucial factor in our field of work; knowing that the PagerDuty platform holds my teams accountable in response scenarios gives us the peace of mind we need to focus on our work, both on-the-ground and internal operations.” said Knauss. If mTOMADY is unable to respond to inquiries in a timely manner, it could mean that someone out there isn’t getting the care or support they need.

Healthcare in the Sahara During the COVID-19 Outbreak

Since the outbreak of COVID-19, mTOMADY has drastically changed the way it operates: They have canceled or delayed all non-essential meetings until further notice and have moved from much of their on-the-ground work to a predominantly digital approach.

However, mTOMADY is also constantly looking for ways to offer assistance. This includes working directly with rural hospitals to provide support and shift their focus to COVID-19-related response by distributing protective equipment to local and rural communities to help slow the spread of the virus. Looking ahead, mTOMADY wants to create a way to integrate PagerDuty into their global health emergency systems so they can detect spikes in illnesses or outbreaks in both local and rural communities.

Looking Into the Future

mTOMADY plans to continue expanding its use of PagerDuty into different facets of the business and go beyond just on-call management. They plan to integrate PagerDuty into their emergency response processes, including integrating with their ambulance service that works directly with rural communities. Additionally, they are looking to expand PagerDuty into their supply and equipment networks so that when there is some sort of communal shortage, they are alerted in real time and can act accordingly.

To learn more about how PagerDuty is working to provide resources to healthcare companies, specifically around the COVID-19 outbreak, visit our COVID-19 resources page. Additionally, visit PagerDuty.org to learn how PagerDuty partners with organizations to deliver on their mission when moments matter, including providing Impact Pricing to organizations like mTOMADY.

The post mTOMADY: Expanding the Reach of Healthcare to Underserved Countries appeared first on PagerDuty.

But in the last few years, advancements in technology have opened the door to new opportunities that can change the way we think about doctor visits and in-person healthcare. Parsley Health, a holistic medical practice that offers online medical care and consultation in almost all 50 states, is helping lead the next evolution of healthcare.

As part of PagerDuty’s commitment to helping organizations scale to meet acute and changing needs in healthcare in response to COVID-19, we are offering healthcare organizations like Parsley Health 20 free PagerDuty licenses for 6 months.

Healthcare From Anywhere

Parsley Health recognizes that going into the doctor’s office may not be the ideal choice for many people, either due to scheduling conflicts or fears tied to seeing a doctor, and they are one of the pioneers of using telemedicine to build strong relationships with their members and patients.

Using the Parsley Health platform, members can meet with medical providers and health coaches through uninterrupted, online one-on-one sessions, either via phone calls or video calls. These sessions may include assessing health risk factors, writing prescriptions, stress management techniques, and even creating comprehensive plans around dietary changes and mental health improvement.

The benefits are twofold: Patients can engage in online sessions within the comfort of their own homes, and providers experience reduced levels of stress and more genuine conversation. These interactions help the providers build rapport and strengthen their relationships with patients.

In addition to personalized one-on-one sessions, Parsley Health also offers members an online patient portal that allows patients to securely contact their doctor via messaging 365 days a year. They also offer a proprietary digital questionnaire to monitor and track patient progress over time. Presented to patients before meeting with a doctor, the questionnaire is scored according to the Parsley Symptom Index (PSI) and gives providers a better understanding of a patient’s symptoms and overall health over the past two weeks. The information is then used to make recommendations regarding a patient’s individual healthcare needs.

The Impact of Telemedicine During Global Health Crises

While the use of telemedicine has become more accessible and widely adopted over the last several years, the benefits and advantages have been especially illuminated during the COVID-19 outbreak. “Through telemedicine, doctors can evaluate patients’ symptoms and assess whether they need to be seen in person, preventing the unnecessary exposure of both parties to COVID-19. It also allows doctors to provide continuity of care to their patients—other medical needs do not stop in the face of a global health crisis,” shared Martín Beauchamp, Manager of Infrastructure, Security, and Data at Parsley Health, whose team is responsible for the alerting infrastructure and incident management operations that keep the platform up and running.

Because of the increased accessibility to technology and the fear for some at-risk individuals to go out in public, telemedicine can play a pivotal role in maintaining the continuity of personal care during health crises. “With more people turning to telemedicine during this global pandemic, a whole new audience is learning how easy and helpful online care can be,” said Beauchamp. “Our hope is that people who have lived in places where there is a more limited range of medical services finally have access to the care they need.”

“Telemedicine has a unique, dual-role to play during COVID-19: Protecting frontline medical workers and serving the needs of patients in the community,”
– Martín Beauchamp, Manager, Infrastructure, Security, and Data, Parsley Health

For Parsley Health, the organizational effects of COVID-19 have been seen mainly through an increase in inquiries and questions from members about the virus and what preventative steps they can take to stay healthy. “With the growing uncertainty and constantly changing information around COVID-19, our providers have seen a huge uptick in online messaging with patients who have questions about symptoms, testing, and practical living,” Beauchamp explained.

The Advantages of Being Cloud-Native

With all of these managed services comes the need for a highly available and reliable platform that members can depend on for care, no matter the time or location. High uptime and resilient services are critical to keeping members healthy, happy, and informed. “When the right engineers can respond quickly to issues with necessary context, we are able to meet the high expectations that our members have in every interaction with Parsley Health providers,” explains Beauchamp.

In order for organizations to keep pace with technology in society, they need to find a way to adapt to the new ways people consume and use products and services. Historically speaking, healthcare typically lags in innovation compared to other more cloud-forward industries. With challenges like private data, compliance, and other regulatory requirements, digital transformation is thought of as a long-term game with paced, incremental advances.

Parsley Health is an innovation outlier in the healthcare industry. As a cloud-native company, technology has been embedded into their offerings since day one. They built a microservices-based platform within Kubernetes and orchestrate their containerized environment through Google Kubernetes Engine (GKE). On the front end, the engineering team uses tools like React and GraphQL, which enable developers to be extremely productive.

With this highly available and forward-looking technology stack, Parsley Health is able to offer premium products and services to their members in real time, around the clock. But in order to keep these services stable, they needed a platform that could keep a pulse on their digital environment.

Using PagerDuty, Beauchamp’s team is able to rally the right people at the right time during incidents and take a prescriptive and calculated approach to incident remediation. “PagerDuty is a crucial part of this process, because, as we receive alerts from monitoring tools, it allows us to respond with the most appropriate personnel based on the situation at hand,” shared Beauchamp.

With PagerDuty’s ecosystem of over 350 integrations, his team has the ability to centralize tools like Slack and Datadog into one single point of ingestion to ensure monitoring data and incident communications are visible and easily accessible to key stakeholders during the lifecycle of an incident. “PagerDuty continues to help us keep our mean time to resolution as low as possible, rally the right resources during incidents, and provide high level visibility for a distributed team,” said Beauchamp.

Since implementing PagerDuty, Parsley Health has seen several benefits, including:

• Improved reliability and uptime, leading to a positive user experience for members
• Better visibility into an incident’s makeup, which helps distributed teams escalate the issue to the appropriate personnel quickly and efficiently
• Improved MTTA/MTTR through automated escalation within PagerDuty and real-time communication through Slack

PagerDuty partners with Parsley Health to ensure their services are always online and available for their users, even when seconds matter.

“Parsley Health will continue to rely on PagerDuty to give us timely alerts so that engineers can respond to operational issues. Together, we’ll keep the platform healthy so our providers can keep our members healthy.”
– Martín Beauchamp, Manager, Infrastructure, Security, and Data, Parsley Health

Visit PagerDuty.org to learn more about PagerDuty’s investment in Time-Critical Global Health and how we’re helping bring essential care to people who need it, in the hardest-to-reach places, in the moments they need it most. Additionally, check out our COVID-19 resource center to see how we’re working with companies and offering resources to help combat the COVID-19 outbreak.

The post Parsley Health: Bringing Telemedicine to the Front Lines appeared first on PagerDuty.

PagerDuty customers primarily leverage our platform for coordinating an incident response process when technical issues happen, such as a bad deployment, network degradation or failed hardware. Many of our customers have also realized that this incident response process can be applied to other business-impacting incidents, and have begun to use PagerDuty for other use cases, such as working with high-profile customer support tickets, security issues, or even emergency situations like what we’re encountering with the COVID-19 outbreak.

As part of our crisis communication plan, we identified several scenarios that would benefit from leveraging PagerDuty to expedite our ability to respond to major incidents at PagerDuty:

• Notify our Crisis Communications team that there is potentially an urgent issue that requires their attention.
• Update our employees on the status of a major incident.

Scenario 1: Notifying Our Crisis Communications Team of an Urgent Issue

It’s critical that our Crisis Communications team is made aware of business-impacting incidents so that we are able to minimize internal disruption and to make sure that we can keep external stakeholders informed. Our Crisis Communications team is a group comprised of folks from our People Operations, Executive, Legal, Marketing, and Facilities teams.

Within PagerDuty, the configuration for notifying the Crisis Communications group is relatively straightforward.

• We have a service and escalation policy that’s dedicated to this group. It has an email address associated with it so that folks can easily trigger an incident via email (as well as through our mobile or web applications).

• All members of the Crisis Communications team are set up on the first level of an escalation policy, and all have multiple notification rules that are set up to notify them immediately should an incident get triggered. Note: It’s important that all users have multiple notification rules that will notify them immediately for redundancy purposes, as well as to ensure that they get notified of the incident even when one of their peers acknowledge the incident before they receive their notification.
• The service is also configured with a conference bridge, which helps facilitate getting the team together to resolve issues in real time via tools like Zoom.
• Our Slack integration is used to keep stakeholders up-to-date in a private Slack channel.

Scenario 2: Keeping Our Employees Up-to-Date on a Major Incident

For major events like the COVID-19 outbreak, it’s important to ensure that we are communicating with all of our employees as the rapidly evolving situation changes. Given that PagerDuty is a global company, we designed a configuration in which we can communicate with each region as needed. Below are some details on how it’s set up.

• We set up two services for each region: one for connecting with leadership and one for communicating with all employees. We also have services set up to communicate with our executive and senior leadership teams.
• Each service has an email integration so that incidents can be triggered via email, or through our web application or mobile app.
• We have three levels of escalation for each region. These folks are expected to facilitate and coordinate a response to each incident, similar to the Incident Commander’s role for a technical incident.
• All employees in each region are set up on a team.
• Each service is also configured to automatically run a response play on ticket creation. The response play is configured to add the regional team as stakeholder users, and they will immediately get notified when there is an update.

• As the situation progresses, the incident owners will send out status updates, which will notify all employees (subscribers).
• Once the incident is resolved, the incident owner will resolve the PagerDuty incident.

Through these two workflows, we can ensure that we can swiftly and efficiently get the right team on the issue, and can keep all affected parties up to date. Should you have any questions or need any help configuring your PagerDuty account to enable your team to respond to critical issues, please contact us at support@pagerduty.com.

The post How We Use PagerDuty for Emergency Response appeared first on PagerDuty.