Query data directly – The magic of Virtualized Views

In today’s world, data is everywhere but accessing data can be a huge challenge. Assume we want to obtain Market data (LMP Or Market Awards) or data from PI. This typically means that someone or an application has to do the following – 

• Connect to Source (ISO or PI using certificates and/or credentials). If data is in an API, connecting may be more complicated because certificates and ISO-specific methods are required.
• Access Data – If data is in an API data has to be queried via a rest interface
• Save Data to new source – Data has to be stored in a file or specific data store
• Query data from the new source

With the magic of virtualized views, users can simply click one button to query data directly from the source. Agile Analytics takes care of the connection and access  (regardless of whether the source is an API, file, database, or data lake) and the virtual view technology means you don’t need any data storage. 

How simple is it to actually view the data? Well, users can go and select columns from the data and put in filters using a front end. Alternatively, they can also write direct SQL Queries. However, non-technical users can also use a Chat-GPT-like interface to ask questions in plain English such as “Give me all the units and hours with market awards greater than 10 MW”, eliminating the need for technical knowledge.

This becomes a game-change for clients because data can now be accessed within a few clicks using plain English (no coding or technical knowledge involved). 

We can also look at more complex scenarios now that require greater storage. These solutions can typically be created in an hour.

For example, many clients use our system to create ISO data lakes. The system can connect to almost 10,000 APIs from PJM, MISO, SPP, ERCOT, CASIO, ISO-NE, NYISO, etc. and can query and store data. API will be queried every 5 minutes, at hourly, or daily intervals as required and stored. The system can also automatically re-query any missed data if the ISO was late in publishing data. 

Steps  – How to Create an Automated ISO Data Lake in under an hour

You have just created an entire ISO data lake with automation that will keep it updated with data from the ISO – All within under an hour.

We can do the same process with PI data. Let’s say we need to analyze Wind output and curtailments by reviewing PI data. 

Steps  – How to Create a PI Data Lake in under an hour

Combine the steps above to create an even more powerful solution. 

Once you’ve done this, now we can combine PI and Market data and analyze it to help solve deeper problems. For example, if the ISO is curtailing your wind units and you are not sure if the curtailments are for economic or other reasons, the market and PI data can be reviewed to ensure that the market is compensating you properly depending on the type of curtailment. All this can again be accomplished in a matter of hours or days vs months with traditional/legacy tools. 

Quick Summary of the Solution

1. Query ISO or PI data  DIRECTLY with a few clicks by writing questions in plain English
2. Build more sophisticated solutions as an ISO data lake in HOURS
3. Build user application-like dashboards to help with decision-making. The dashboard can automate joining PI, ISO data and running logic to calculate metrics used in dashboards.

Now, the question is, can you also accomplish this with the tools you have today? Absolutely – but this will take IT resources and almost a month with current tools. Why spend so much money and effort when you can create an even better solution with Agile Analytics in just one day or less?  Contact us at info@rpm.104.myftpupload.com to be added to our mailing list or to get a demo of how Agile Analytics has enabled clients to easily and quickly create meaningful Analytics solutions to drive decision-making and improve efficiency.