Demand side management in electric vehicles for smart charging and discharging based on state aggregation and q-learning

C. R. Rajesh1* and Aravind S. P.2*

*Correspondence:
C. R. Rajesh,
crrajesh99@gmail.com
Aravind S. P.,
sparavind@live.com

Received: 24 August 2023; Accepted: 21 October 2023; Published: 04 January 2024.

License: CC BY 4.0

Copyright Statement: Copyright © 2024; The Author(s).

Use of traditional energy sources results in significant pollution. International organizations are making many efforts to reduce emissions of carbon dioxide (CO2). According to research, EVs can reduce CO2 emissions by 28% by the year 2030. However, the prohibitive price of EVs and the scarcity of outlets for charging continue to be two of the biggest barriers to the widespread use of electric vehicles. In this paper, a detailed demand-side management approach for a network-connected, solar-powered electric vehicle charging station is provided. The proposed approach reduces the requirement for conventional power sources and addresses the current problem of insufficient EVCS by using a solar-powered EVCS in order to make up for the electricity used during peak demand. Models for PV power plants, industrial loads, residential loads, and charging stations for electric vehicles were created using the real-time data. Additionally, a method based on deep learning was devised to control the microgrid’s supply of electricity and to recharge the battery of the electric vehicle during off-peak hours. Two alternative machine learning techniques for figuring out the level of charge in a device that stores electricity were also put to the test. Finally, a 24-h case study using the suggested demand management system structure was conducted. The data show that peak consumption is offset by using a charging station for electric vehicles during peak periods.

Keywords: microgrid, EVCS, solar-powered electric vehicle, PV power stations, commercial loads

1. Introduction

Numerous environmental organizations have established carbon dioxide (CO2) reduction programs and policies. Two of the most intriguing possibilities being investigated as remedies for the world’s expanding environmental issues and energy supply difficulties are the utilization of sources of renewable power and the electrification of transportation networks. As a result, the production of electricity and the networks that support travel are shifting to sources of renewable energy and electric vehicles (EV) (1).

On both the supply and demand sides, electrical energy management is used to reduce energy expenditures and alter load patterns. Enhancing the operational effectiveness of energy production, transmission, and distribution is the goal of supply side management (SSM). SSM has many advantages, such as enhancing customer value by ensuring efficient manufacturing of energy at the lowest feasible cost, satisfying electricity demand without the need for new infrastructure, and limiting the impact on the environment. The volatility of fuel prices, however, has an impact on supply-side administration because of the methodology it uses to manage thermal producers (2).

Massive electric vehicle integration into power systems can improve the system’s stability and low-carbon transformation (3). In the last few years, there has been a significant rise in the advancement of model-based optimization techniques that simulate the EV dispatch issues for dependable travel patterns via local charging facilities and different ancillary services via V2G technology, such as asymmetry of energy service, carbon dioxide assistance, electrical support, the frequency oversight, and so forth.

Pay-per-kWh and pay-for-time are the two pricing structures that energy-sharing systems most typically employ. On hybrid systems, the suggested dynamic pricing technique works well when applied to the computation of tariffs. This has many benefits, including increased voltage stability, higher power factor, and profit maximization. It is intended to be a time-to-use policy without a mechanism for electricity usage during peak hours (4).

2. Related works

A neural network is told to use reinforced learning to learn from past data in order to provide the best answer with imitating the system’s unpredictability. Deep neural networks and reinforcement learning are widely used in computer vision models, and they produce good results (5). The development of EV technology is being pushed forward by governmental regulations and environmental laws. Dynamic rates for EV scheduling, for instance, will be made possible by cost savings and claims that the Chinese authorities offer tax breaks to drivers of electric cars in the nation’s capital. (6).

In addition to taking legislative measures to lower battery costs, the Indian government provides subsidies for the construction of charging facilities. Such a program could offer more affordable maintenance for an EV acquisition as the power source is probably the most expensive part of an EV. By 2020, EV charging infrastructure will essentially fall into two kinds. Level 1–2 is the first category; it uses alternating electricity, requires 3–4 h to fully charge, and has a range of up to 150 miles. Level 3–4 vehicles, which can drive up to 200 miles and require a full charge in 1–2 h and 15–45 min, make up the third tier. (7).

The charge timetable takes into account a number of variables, including arrival time, range anxiety, and traffic capacity (8). A variety of private companies are expanding their infrastructure in order to implement ultra-fast charging stations, including EVho, Tesla, KIA, charger point, and others. For instance, the United States had around 43,000 charging stations in June 2016; many businesses have offered electric vehicle charging maps to assist clients in finding a charging station.

Despite an increase in charging station availability, range anxiety remains a significant issue in EV scheduling (9).

For instance, they could set up roof solar panels and, during periods of high demand, share additional energy with the smart grid to use solar energy for refilling electric vehicles. The two pricing models that energy-sharing systems most typically employ are pay-per-kWh and pay-for-time. With tariff computation on hybrid systems, the recommendation for dynamic pricing technique was applied successfully. This has numerous advantages, including improved power factor, higher electrical stability, and maximizing profits. The planned network thus gives users a financial incentive to deploy EVs and also offers quick charging (10).

3. Proposed method

. We suggest a dynamic pricing strategy that schedules the charging of electric vehicles during times of low demand (when electricity is less expensive) and surplus energy discharge during peak hours (when electricity is more expensive). As a result, the suggested network offers consumers an economic reason to install EVs while also providing speedy charging (Figure 1). This has various benefits, such as increased power factor, increased profit, and improved voltage stability.

FIGURE 1
www.bohrpub.com

Figure 1. Energy communication diagram between first stage and second stages.

3.1. ECS modeling

It is intended to be a time-to-use strategy without a mechanism for electricity usage during peak hours. As a result, the suggested network offers consumers an economic reason to install EVs while also providing speedy charging.

ARMA models were applied to predict stagnant time-series data. The ARIMA model outperforms the ARMA model for dealing with unpredictable time series data. Currently, augmented ARIMA is employed to predict ever-more complex systems that can derive characteristics from a wide range of variables (61). On load patterns and profiles, this has a significant effect (60). Ann-based techniques can be used to handle this kind of issue, and accurate forecasts may be created without sacrificing precision.

Figure 2 shows the block diagram of multiagent Markov decision problem formulation with cooperation criteria. Initially, the input nodes are given to the input layer. Then this input layer split it and sent it Deep Neural Network. From DNN it is forwarded to Dynamic Pricing unit and then finally sent to the output layer.

FIGURE 2
www.bohrpub.com

Figure 2. Dynamic policy approach for the formulation of multi-agent Markov decision problems with cooperative requirements.

3.2. Scheduling by queuing criteria

In the concept of a queue, we categorize the scheduling issue. A plain joint-the-shortest queuing. As a result, the inbound request feels range anxiety. The assignment for each station, multiply N by the amount of queue space and M by the number of active EV requests.

The output of an EV using the suggested queuing mechanism is a state of charge. Each supply station has N comparable sites and is a multi-agent service. Each queue is allotted EV charging/discharging according to a Poisson distribution. As a result, the best scheduling won’t start until the queue is completely empty. Vehicles looking to charge or discharge benefit from the First-In-First-Out (FIFO) system. In order to determine the vehicle charging and discharging cut-off level, it is necessary to take into account both the supply station’s plug availability and EV arrival time.

4. Experimental results and discussion

A nearby real power substation’s statistics on industrial as well as commercial load were used to assist in building the micro grid. Data for a source of energy models were acquired from,1 a publicly accessible internet data source. The suggested method was simulated using MATLAB 2020a. The machine learning-based time series method was used to simulate and forecast load models for residential as well as business loads over a time period utilizing the collected data.

From this, we can know about the various variables for the energy sources. Figure 3 shows the Correlation between the different variables of the energy source. Figures 4, 5 show the EV charging station and solar power station for 24 h correspondingly. Both are increased rapidly and decreased gradually. From the above figures, we can conclude that both will increase gradually.

FIGURE 3
www.bohrpub.com

Figure 3. Correlation between the different variables of the energy source.

FIGURE 4
www.bohrpub.com

Figure 4. Power profile of EV charging station for 24 h.

FIGURE 5
www.bohrpub.com

Figure 5. Profile of solar power station for 24 h.

5. Conclusion

This study develops a perfect EV scheduling strategy with dynamic pricing. The suggested MADNN optimum scheduling gives the EV consumer more incentives and lowers their electricity costs. The network programming approach that has the quickest network convergence is supported by these findings. It also controls the length of time an EV must wait by implementing a queuing system in relation to other EVs entering at EVSS.

Footnotes

  1. ^ https://data.nasa.gov/dataset

References

1. Da Silva FL, Nishida CE, Roijers DM, Costa AH. Coordination of electric vehicle charging through multiagent reinforcement learning. IEEE Trans Smart Grid. (2020) 11:2347–56.

Google Scholar

2. Mahfouz MM, Iravani MR. Grid-integration of battery-enabled DC fast charging station for electric vehicles. IEEE Trans Energy Convers. (2020) 35:375–85.

Google Scholar

3. Aljohani TM, Ebrahim AF, Mohammed OA. Dynamic real-time pricing mechanism for electric vehicles charging considering optimal microgrids. IEEE Trans Ind Appl. (2021) 57:5372–81.

Google Scholar

4. Das S, Acharjee P, Bhattacharya A. Charging scheduling of electric vehicle incorporating grid-to-vehicle and vehicle-to-grid technology considering in smart grid. IEEE Trans Ind Appl. (2021) 57:1688–702.

Google Scholar

5. Dong C, Chu R, Morstyn T, McCulloch MD, Jia H. Online rolling evolutionary decoder-dispatch framework for the secondary frequency regulation of timevarying electrical-grid-electric-vehicle system. IEEE Trans Smart Grid. (2021) 12:871–84.

Google Scholar

6. Fang C, Lu H, Hong Y, Liu S, Chang J. Dynamic pricing for electric vehicle extreme fast charging. IEEE Trans Intell Transp Syst. (2021) 22:531–41. doi: 10.1109/TITS.2020.2983385

CrossRef Full Text | Google Scholar

7. Malek YN, Najib M, Bakhouya M, Essaaidi M. Multivariate deep learning approach for electric vehicle speed forecasting. Big Data Min Anal. (2021) 4:56–64.

Google Scholar

8. Jeyaraj PR, Nadar ER. Computer-assisted demand-side energy management in residential smart grid employing novel pooling deep learning algorithm. Int J Energy Res. (2021) 45:7961–73.

Google Scholar

9. Liu Y, Liang H. A three-layer stochastic energy management approach for electric bus transit centers with PV and energy storage systems. IEEE Trans Smart Grid. (2021) 12:1346–57.

Google Scholar

10. Said D, Mouftah HT. A novel electric vehicles charging/discharging management protocol based on queuing model. IEEE Trans Intell Veh. (2020) 5:100–11.

Google Scholar


© The Author(s). 2024 Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.