The Power of Pilots

Chen Linchevski, Precognize, Israel, proposes that a well-planned technology pilot, rather than a direct request for proposal, builds stronger foundations for a successful predictive analytics technology roll-out.

While new technology promises a bright future, uncertainty remains over its ability to deliver on this pledge. According to a McKinsey report, many new technology implementations fail extravagantly, with companies unable to capture value from 70% of their technology pilots. In order to mitigate the risk of new technologies, many oil and gas executives turn directly to requests for proposals (RFPs). The theory goes that by forcing technology vendors to present themselves in a standard format, the company can determine the best company or technology to meet its needs.

However, theory and reality do not always match. Based on experience with scores of pilots and deployments of predictive analytics solutions in oil and gas and other process industries, it is still premature to jump into an RFP process. Much of the oil and gas industry is still in the pr e-pilot phase. While it sounds counterintuitive, this article recommends that RFPs are dropped and, instead, the focus should be on building and implementing informative pilot processes. RFPs should be saved for after some experience with the technology has been gained, and then, when the time comes, enough necessary knowledge will be available to scale up and embrace the new predictive analytics solution.

How to prepare for a successful pilot

When planning a pilot, it is important to be aware that even limited pilots have a tendency to fail. There are five concrete recommendations to ensure that predictive analytics pilots bring substantial value to oil and gas companies, and have the potential for an even more successful company-wide roll-out:

Understand a plant’s needs

Different plants have different needs. It is crucial to dig deep into the past performance, challenges, and achievements of the plant before planning for the future. Consider previous and current business challenges, and use this understanding to generate key performance indicators (KPIs) for the pilot. For example, is the aim to reduce equipment failures, improve overall production quality, increase output, or is there a different goal?

For example, in the early days of Precognize, it was called to help a global chemical company with a 12-month European pilot. The pilot goal was to detect equipment failures; however, the pilot’s champion neglected to mention that the plant had not had any major failure in the past five years. The customer feedback was that they loved the technology, but decided not to continue with it because they could not determine any value that it provided for them. This was the result of selecting the wrong KPI.

When most people talk about predictive analytics, they are thinking about mechanical failures. In reality, the majority of the value a predictive analytics solution generates is derived from the day to day improvements it delivers in availability, output, and other ongoing metrics, e.g. if a valve is only partially malfunctioning, a reduction in output but not in availability may be observable. Once this is fixed, productivity and thus OEE will improve. That is why predictive analytics are not just about knowing when to replace the heat exchanger, for example. They are about generating day to day value from the increased visibility the plant can gain.

Pilot KPIs should be numerical, and reality-based. A customer once informed Precognize that its current availability is 86%, and they want to bring it up to 90%+. However, once their availability was measured, it was discovered that their current baseline was actually much lower than 86%; yet it was still possible to increase their availability by 4%. The KPI’s baseline must be agreed upon in order to measure improvements appropriately.

Due to the difficult yet crucial nature of selecting the right KPIs, the first stage for any pilot has to be impact analysis. This is a collaborative process between the vendor and the customer, to determine which issue(s) are of the greatest concern, and what value they would like to realize from the technology implementation.

Do not turn the pilot into a theoretical exercise

Although numbers are crucial, it is equally important not to get distracted into turning the entire pilot into a statistics test. As part of the initial weeding out process, pilot managers often waste precious time by sending historical data to potential vendors, asking the

vendors to find the failure that occurred. At a recent conference, the vendor of a large oil and gas company demonstrated his statistical model, using past events to deduce what would occur in the future. When asked how many events they draw on in their predictive modelling, he admitted to just two events, which leads to a very poor prediction model. So even if this vendor found a previous problem in the data sent to them, it would not indicate a high ability to predict future events successfully.

Testing a vendor based on past failure data is a poor indicator for how well they may succeed in predicting future issues. It does not indicate how quickly they would determine the root cause of a failure while it is in progress, or how many alerts they would flag if something similar happens in the future, which are both important questions to ask. Identifying a past failure from historical data is still an important capability, but it is something that every vendor should be able to achieve. Another way needs to be found to truly assess their capabilities, one which focuses on what provides value in a specific predictive analytics solution and does not rely on a simplistic checkbox exercise.

Focus on operations, involve IT

Contrary to what many pilot managers seem to believe, at the pilot stage, predictive analytics is not primarily an IT project. It is important to ensure that operations is involved and actually leading the charge at this juncture. Many RFPs and tenders are clearly led

by IT. However IT, quite rightly, is primarily concerned with scaling. Operations is more involved in the smooth functioning of the plant, typically based on monitoring and reducing incidents, and avoiding disruption to an effective system.

From IT, it is common to see questions like “Does your system work with containers? Does it work with virtual machines, or with Azure?” But if operations are consulted, questions such as the following are asked: “How many alerts do you issue per day? How much of the plant do you cover? What types of problems can be discovered using this technology? Who are the target users? How much of their time will it take each week?” Although IT’s questions are important, it is the operations questions that are most relevant to a pilot’s success. It is vital for IT and operations to work together for what the McKinsey article calls the effective IT/OT ‘last mile’ convergence to ensure the success of a predictive analytics pilot.

Set the right time period

Before a pilot commences, the appropriate amount of time needs to be allocated for it. According to the McKinsey report, there are some companies that spend one or even two full years on a pilot. This is unreasonable in most cases. Pilots should ‘fail fast’, to borrow a mantra from the startup world. The faster a pilot is completed the faster it can either be scaled up to the rest of the company, or killed before moving on to something else. Drawing on experience working with oil and gas companies, Precognize has determined that the ‘sweet spot’ for predictive analytics pilots is approximately three months.

A pilot timeline should look something like this:

• Impact analysis: 1 – 2 weeks.
• Modelling workshop: 2 weeks.
• Training: 1 hour per day for 1 week.
• 3 months of managing the pilot, including an average of 0.5 – 1 hour per day of engineer time, backup support for reviewing alerts, and operations and IT involvement when necessary.

Assign appropriate human resources

The other key resource for a pilot is human resources – setting the right people to run the pilot and use the system, so that they can gain value from it. In just about every plant, the employees already have 100% workload, if not more. Without the right people dedicating enough time to the pilot, it is a recipe for failure.

It needs to be decided which people in the plant need to access the software on a regular basis, how much time they would spend using it, what happens if the software sends an alert, and who responds to alerts. From experience, at a minimum, the process engineer needs to review every alert. Depending on the alert, he/she also needs access to other managers in order to investigate it more thoroughly, but the right person for the job varies depending on the plant.

What is most important is that the person primarily responsible for the pilot has thorough knowledge of the plant, rather than the best computer knowledge. In one implementation in Bulgaria, the primary pilot manager was a shift leader, working together with the process engineers. This turned out to be the magic formula to get the job done.

A well-planned pilot can be a step towards Industry 4.0

Although it is tempting to jump straight to an RFP, there is a lot to be gained from a pilot. The challenge is to run a pilot in a way that develops into a next stage, whether that is an RFP, another pilot, or the decision not to implement the technology at this stage. When planning a pilot, take care in identifying the right KPIs and allocating the right time and

resources, asking vendors the right questions, and focusing on operations. The value that predictive analytics technology can bring to your company can quickly and easily be evaluated, and moves will be made towards the goal of integrating Industry 4.0.

Reference

1. NARAYANAN, S., and COXON, M., ‘It’s the last IT/OT mile that matters in avoiding Industry 4.0’s pilot purgatory’, McKinsey & Co., https://www.mckinsey.com/business-functions/operations/our-insights/operations-blog/its-the-last-it-otmile-that-matters-in-avoiding-industry-40s-pilot-purgatory