February 23, 2022

February 23, 2022

February 23, 2022

Coordinating the distributed energy future: Lessons from the NYISO

Coordinating the distributed energy future: Lessons from the NYISO

Coordinating the distributed energy future: Lessons from the NYISO

Connor Waldoch, Former Senior Manager, Policy & Regulatory Affairs

Connor Waldoch, Former Senior Manager, Policy & Regulatory Affairs

Connor Waldoch, Former Senior Manager, Policy & Regulatory Affairs

Connor Waldoch, Former Senior Manager, Policy & Regulatory Affairs

Connor Waldoch, Former Senior Manager, Policy & Regulatory Affairs

Connor Waldoch, Former Senior Manager, Policy & Regulatory Affairs

Since market operators only have insight into the transmission system, local distribution utilities and DER operators play a key role in ensuring that distributed resources are operated in a manner by which distribution systems, in addition to the bulk power system, remain efficient and reliable.

How are communications currently carried out between distribution utilities and the NYISO?


As more distributed resources have been deployed in New York, enrollment in utility demand response (DR) programs has grown. However, current DR activations have a spotty record of efficient coordination between utilities and the NYISO, in part due to a lack of processes on the NYISO side. In Q3 of 2021, utilities activated DR on ten days, and the NYISO also utilized Supplemental Resource Evaluations (SREs) on three of those days in order to meet statewide capacity needs.


SREs represent resource commitments outside of the normal Security Constrained Unit Commitment (SCUC) and Real-time Commitment (RTC). They only address resource deficiencies - ignoring costs. However, during a presentation of 2021 Q3 results, the NYISO’s market monitor noted that some of the SREs were probably unnecessary, and would not have been utilized had the ISO considered utility DR activations in their SRE process (which it currently does not). The peak utility DR deployment was nearly 650MW, large enough to rank in the top fifteen largest generating units by MW capability per the 2021 Gold Book, and in the top five when only considering downstate zones.


Despite the size of this resource, the NYISO has neglected to integrate it into its daily planning process. This lack of coordination on a relatively simple mass utility DR deployment portends trouble in implementing effective communications for the much more geographically and technically diverse array of resources anticipated under the DER model.


The coordination medium is also important. Ideally, a systems integration would allow for machine-readable signals to readily travel between a DER Aggregator, Distribution Utility, and the NYISO. Current processes do not always support this style of automated system. Instead of relying on human-driven communications, such as email or phone, only as a backup, certain processes utilize those methods as a starting point. Section 6.7.3 of the Transmission and Dispatch Operations Manual describes the maximum generation pickup process as:


“At the NYISO’s judgment, generators will be instructed via voice communication to increase output to their upper operating limits as soon as possible until directed otherwise.”


Voice communications are typically used to rapidly relieve a transmission violation outside of the Real-Time Dispatch (RTD) cycle, but rather than relying on the same Automatic Generator Control (AGC) systems used for normal operations, these ad hoc voice communications are the default. While the timing of an event may be outside of the regular automated control process, it should not preclude the use of machine signal communications for the pickup.


Similarly, there are potential communication issues related to DER outages and availability. Currently, the NYISO’s 2222 filing contemplates a process by which the distribution utility (DU) will review an Aggregator’s dispatch plan from 12:00 PM up until 10:00 PM the day prior to dispatch. If the system conditions have changed, the DU will “advise the Aggregator as soon as practical.”


The figure below shows how this process fits into the normal day-ahead flow. Under this process, the time it takes a DU to confirm an Aggregator’s schedule may be nearly twice what it takes the NYISO to optimize the entire market. Should the DU determine a DER may not fulfill its NYISO schedule, the Aggregator must inform the NYISO of a derate and then submit bids in real-time that reflect the modification of their schedule.



Under the proposed coordination process, DUs can inform aggregators of schedule disruptions up to only two hours prior to the market day, forty-six fewer hours than necessary for traditional generators to schedule an outage without penalty. Forced outages and derates directly reduce the amount of capacity resources are qualified to sell by contributing to their derating factor. Derating factors are calculated differently for DERs compared to traditional generators, and specific methodologies within DERs depend on resource type and whether an Aggregation is heterogeneous or homogeneous. Regardless, the calculations generally rely on resources to be available to meet their day-ahead schedule.

Additionally, it follows that modifications to day-ahead schedules would arise from differences in system topology that also affect real-time operations. While the NYISO’s filing requires that DERs modify real-time bids in response to DU requests, real-time communication following a schedule disruption is not contemplated in the filing. Following the above timeline, Aggregators will perpetually be in a situation in which exogenous distribution network factors may impact their ability to meet day-ahead schedules from the NYISO, with no prescribed recourse in terms of real-time coordination.


The traditional outage scheduling process already relies on substantial manual coordination and information sharing back and forth between TOs, the ISO, and generators. The addition of a DU-Aggregator communications layer, shortened timeline, and likelihood for DER resources to greatly outnumber traditional generators suggests a dramatic increase in the workload to coordinate these human-driven information requests and submissions.


Exposure to penalties and a reduced ability to sell capacity is a real risk for DERs. The competitive nature of this market structure is also at risk. If the processes remain oriented towards last century’s resource model, then the market will provide those resources preferential treatment and fail to obtain a competitive balance.


Finally, there is the simple issue of insight into the distribution system in order to maintain a reliable and well-managed grid. By fully engaging and integrating with the DUs wherever possible, the NYISO will increase visibility into distribution system operations as the distribution-side resource base continues to grow. Promulgating communications standards for the next wave of DERs will shift those costs from the ISO to developers as they meet the standards set by market rules.


Relying on traditional communications infrastructure and layers of manual coordination will not incentivize DERs to adopt advanced communication standards. Simply put, investments in automated coordination and communication standards by the NYISO now will pay off substantially in avoided coordination costs in the future.


These are only a few of the potential issues standing in the way of full and fair participation of DERs in a wholesale market, non-exhaustive of coordination issues relevant only within NYISO. We will continue to explore the facets of coordination as well as the other key issues as identified by FERC in the NYISO and nationwide.


Next time: we will unpack specific issues related to distributed resources, 2222’s impact, and recent filings.

Since market operators only have insight into the transmission system, local distribution utilities and DER operators play a key role in ensuring that distributed resources are operated in a manner by which distribution systems, in addition to the bulk power system, remain efficient and reliable.

How are communications currently carried out between distribution utilities and the NYISO?


As more distributed resources have been deployed in New York, enrollment in utility demand response (DR) programs has grown. However, current DR activations have a spotty record of efficient coordination between utilities and the NYISO, in part due to a lack of processes on the NYISO side. In Q3 of 2021, utilities activated DR on ten days, and the NYISO also utilized Supplemental Resource Evaluations (SREs) on three of those days in order to meet statewide capacity needs.


SREs represent resource commitments outside of the normal Security Constrained Unit Commitment (SCUC) and Real-time Commitment (RTC). They only address resource deficiencies - ignoring costs. However, during a presentation of 2021 Q3 results, the NYISO’s market monitor noted that some of the SREs were probably unnecessary, and would not have been utilized had the ISO considered utility DR activations in their SRE process (which it currently does not). The peak utility DR deployment was nearly 650MW, large enough to rank in the top fifteen largest generating units by MW capability per the 2021 Gold Book, and in the top five when only considering downstate zones.


Despite the size of this resource, the NYISO has neglected to integrate it into its daily planning process. This lack of coordination on a relatively simple mass utility DR deployment portends trouble in implementing effective communications for the much more geographically and technically diverse array of resources anticipated under the DER model.


The coordination medium is also important. Ideally, a systems integration would allow for machine-readable signals to readily travel between a DER Aggregator, Distribution Utility, and the NYISO. Current processes do not always support this style of automated system. Instead of relying on human-driven communications, such as email or phone, only as a backup, certain processes utilize those methods as a starting point. Section 6.7.3 of the Transmission and Dispatch Operations Manual describes the maximum generation pickup process as:


“At the NYISO’s judgment, generators will be instructed via voice communication to increase output to their upper operating limits as soon as possible until directed otherwise.”


Voice communications are typically used to rapidly relieve a transmission violation outside of the Real-Time Dispatch (RTD) cycle, but rather than relying on the same Automatic Generator Control (AGC) systems used for normal operations, these ad hoc voice communications are the default. While the timing of an event may be outside of the regular automated control process, it should not preclude the use of machine signal communications for the pickup.


Similarly, there are potential communication issues related to DER outages and availability. Currently, the NYISO’s 2222 filing contemplates a process by which the distribution utility (DU) will review an Aggregator’s dispatch plan from 12:00 PM up until 10:00 PM the day prior to dispatch. If the system conditions have changed, the DU will “advise the Aggregator as soon as practical.”


The figure below shows how this process fits into the normal day-ahead flow. Under this process, the time it takes a DU to confirm an Aggregator’s schedule may be nearly twice what it takes the NYISO to optimize the entire market. Should the DU determine a DER may not fulfill its NYISO schedule, the Aggregator must inform the NYISO of a derate and then submit bids in real-time that reflect the modification of their schedule.



Under the proposed coordination process, DUs can inform aggregators of schedule disruptions up to only two hours prior to the market day, forty-six fewer hours than necessary for traditional generators to schedule an outage without penalty. Forced outages and derates directly reduce the amount of capacity resources are qualified to sell by contributing to their derating factor. Derating factors are calculated differently for DERs compared to traditional generators, and specific methodologies within DERs depend on resource type and whether an Aggregation is heterogeneous or homogeneous. Regardless, the calculations generally rely on resources to be available to meet their day-ahead schedule.

Additionally, it follows that modifications to day-ahead schedules would arise from differences in system topology that also affect real-time operations. While the NYISO’s filing requires that DERs modify real-time bids in response to DU requests, real-time communication following a schedule disruption is not contemplated in the filing. Following the above timeline, Aggregators will perpetually be in a situation in which exogenous distribution network factors may impact their ability to meet day-ahead schedules from the NYISO, with no prescribed recourse in terms of real-time coordination.


The traditional outage scheduling process already relies on substantial manual coordination and information sharing back and forth between TOs, the ISO, and generators. The addition of a DU-Aggregator communications layer, shortened timeline, and likelihood for DER resources to greatly outnumber traditional generators suggests a dramatic increase in the workload to coordinate these human-driven information requests and submissions.


Exposure to penalties and a reduced ability to sell capacity is a real risk for DERs. The competitive nature of this market structure is also at risk. If the processes remain oriented towards last century’s resource model, then the market will provide those resources preferential treatment and fail to obtain a competitive balance.


Finally, there is the simple issue of insight into the distribution system in order to maintain a reliable and well-managed grid. By fully engaging and integrating with the DUs wherever possible, the NYISO will increase visibility into distribution system operations as the distribution-side resource base continues to grow. Promulgating communications standards for the next wave of DERs will shift those costs from the ISO to developers as they meet the standards set by market rules.


Relying on traditional communications infrastructure and layers of manual coordination will not incentivize DERs to adopt advanced communication standards. Simply put, investments in automated coordination and communication standards by the NYISO now will pay off substantially in avoided coordination costs in the future.


These are only a few of the potential issues standing in the way of full and fair participation of DERs in a wholesale market, non-exhaustive of coordination issues relevant only within NYISO. We will continue to explore the facets of coordination as well as the other key issues as identified by FERC in the NYISO and nationwide.


Next time: we will unpack specific issues related to distributed resources, 2222’s impact, and recent filings.

Since market operators only have insight into the transmission system, local distribution utilities and DER operators play a key role in ensuring that distributed resources are operated in a manner by which distribution systems, in addition to the bulk power system, remain efficient and reliable.

How are communications currently carried out between distribution utilities and the NYISO?


As more distributed resources have been deployed in New York, enrollment in utility demand response (DR) programs has grown. However, current DR activations have a spotty record of efficient coordination between utilities and the NYISO, in part due to a lack of processes on the NYISO side. In Q3 of 2021, utilities activated DR on ten days, and the NYISO also utilized Supplemental Resource Evaluations (SREs) on three of those days in order to meet statewide capacity needs.


SREs represent resource commitments outside of the normal Security Constrained Unit Commitment (SCUC) and Real-time Commitment (RTC). They only address resource deficiencies - ignoring costs. However, during a presentation of 2021 Q3 results, the NYISO’s market monitor noted that some of the SREs were probably unnecessary, and would not have been utilized had the ISO considered utility DR activations in their SRE process (which it currently does not). The peak utility DR deployment was nearly 650MW, large enough to rank in the top fifteen largest generating units by MW capability per the 2021 Gold Book, and in the top five when only considering downstate zones.


Despite the size of this resource, the NYISO has neglected to integrate it into its daily planning process. This lack of coordination on a relatively simple mass utility DR deployment portends trouble in implementing effective communications for the much more geographically and technically diverse array of resources anticipated under the DER model.


The coordination medium is also important. Ideally, a systems integration would allow for machine-readable signals to readily travel between a DER Aggregator, Distribution Utility, and the NYISO. Current processes do not always support this style of automated system. Instead of relying on human-driven communications, such as email or phone, only as a backup, certain processes utilize those methods as a starting point. Section 6.7.3 of the Transmission and Dispatch Operations Manual describes the maximum generation pickup process as:


“At the NYISO’s judgment, generators will be instructed via voice communication to increase output to their upper operating limits as soon as possible until directed otherwise.”


Voice communications are typically used to rapidly relieve a transmission violation outside of the Real-Time Dispatch (RTD) cycle, but rather than relying on the same Automatic Generator Control (AGC) systems used for normal operations, these ad hoc voice communications are the default. While the timing of an event may be outside of the regular automated control process, it should not preclude the use of machine signal communications for the pickup.


Similarly, there are potential communication issues related to DER outages and availability. Currently, the NYISO’s 2222 filing contemplates a process by which the distribution utility (DU) will review an Aggregator’s dispatch plan from 12:00 PM up until 10:00 PM the day prior to dispatch. If the system conditions have changed, the DU will “advise the Aggregator as soon as practical.”


The figure below shows how this process fits into the normal day-ahead flow. Under this process, the time it takes a DU to confirm an Aggregator’s schedule may be nearly twice what it takes the NYISO to optimize the entire market. Should the DU determine a DER may not fulfill its NYISO schedule, the Aggregator must inform the NYISO of a derate and then submit bids in real-time that reflect the modification of their schedule.



Under the proposed coordination process, DUs can inform aggregators of schedule disruptions up to only two hours prior to the market day, forty-six fewer hours than necessary for traditional generators to schedule an outage without penalty. Forced outages and derates directly reduce the amount of capacity resources are qualified to sell by contributing to their derating factor. Derating factors are calculated differently for DERs compared to traditional generators, and specific methodologies within DERs depend on resource type and whether an Aggregation is heterogeneous or homogeneous. Regardless, the calculations generally rely on resources to be available to meet their day-ahead schedule.

Additionally, it follows that modifications to day-ahead schedules would arise from differences in system topology that also affect real-time operations. While the NYISO’s filing requires that DERs modify real-time bids in response to DU requests, real-time communication following a schedule disruption is not contemplated in the filing. Following the above timeline, Aggregators will perpetually be in a situation in which exogenous distribution network factors may impact their ability to meet day-ahead schedules from the NYISO, with no prescribed recourse in terms of real-time coordination.


The traditional outage scheduling process already relies on substantial manual coordination and information sharing back and forth between TOs, the ISO, and generators. The addition of a DU-Aggregator communications layer, shortened timeline, and likelihood for DER resources to greatly outnumber traditional generators suggests a dramatic increase in the workload to coordinate these human-driven information requests and submissions.


Exposure to penalties and a reduced ability to sell capacity is a real risk for DERs. The competitive nature of this market structure is also at risk. If the processes remain oriented towards last century’s resource model, then the market will provide those resources preferential treatment and fail to obtain a competitive balance.


Finally, there is the simple issue of insight into the distribution system in order to maintain a reliable and well-managed grid. By fully engaging and integrating with the DUs wherever possible, the NYISO will increase visibility into distribution system operations as the distribution-side resource base continues to grow. Promulgating communications standards for the next wave of DERs will shift those costs from the ISO to developers as they meet the standards set by market rules.


Relying on traditional communications infrastructure and layers of manual coordination will not incentivize DERs to adopt advanced communication standards. Simply put, investments in automated coordination and communication standards by the NYISO now will pay off substantially in avoided coordination costs in the future.


These are only a few of the potential issues standing in the way of full and fair participation of DERs in a wholesale market, non-exhaustive of coordination issues relevant only within NYISO. We will continue to explore the facets of coordination as well as the other key issues as identified by FERC in the NYISO and nationwide.


Next time: we will unpack specific issues related to distributed resources, 2222’s impact, and recent filings.

Get started with grid services

Access energy markets