1. Introduction
Due to growing concerns over the depletion of petroleum reserves and environmental issues, the development of electric vehicles (EVs) has attracted significant attention during the past 20 years. Since then, EVs have grown in popularity as one of the best ways to lessen environmental issues and act as a backup for intermittent renewable energy sources. Considering the EV infrastructure’s rapid expansion, EVs do present a viable alternative to protect the electrical grid’s stability (1, 2).
Due to the power of fast charging being significantly higher than the power of conventional batteries, the increase in load brought on by the EV’s lightning-fast charge may exacerbate the peak-to-valley load inequalities, overflowing the distribution grid and resulting in voltage drops, greater grid loss, and uneven power distribution (3).
The distribution grid, which controls the amount of energy flowing through transmission lines, is the sole way to move electrical energy. Accurately forecasting the potential impacts of substantial grid-connected renewable energy production systems and EV charging infrastructure on network performance (EVCS) is essential (4).
By increasing the equipment’s efficiency, power utilities will be able to resolve any operational problems. A strong infrastructure for charging should be backed up with efficient charging in order to benefit both EV users and the utility (5).
2. Literature survey
Batteries perform the best for common vehicle power requirements, despite the fact that there are numerous sophisticated energy storage systems (ESSs). Batteries continue to have issues including greater thermal heating, higher starting and operating costs, slower recharging and shorter travel distances, decreased power density, and shorter operational lifetime. As a result, a vehicle that merely has a battery as an ESS is not efficient (6).
Due to the reduced range brought on by higher energy consumption rates, drivers may feel range anxiety. Additionally, the batteries’ lifespan is shortened as they are subjected to additional strain. The driver’s range is therefore constrained by the amount of battery power that is available. How well a HEV performs is greatly influenced by a variety of factors, including the vehicle’s weight, drag coefficient, frontal area, and a whole host of others (7, 8).
The research on the virtual power market for EVs and renewable energy presents a fueling management system built around a system of multiple agents (MAS). We looked into a V2G scheduling technique’s “source and load” characteristics in the microgrid (9, 10).
Using sophisticated EV charging and V2G technologies, a model was developed to examine the growth in PV power demand. One proposed approach for achieving quick charging without compromising the aging of lithium-ion batteries is multistage constant current (MCC) charging. The charge levels of 5S-CC (five-stage-constant current) charging (a sort of MCC charging) were optimized in this study employing a more efficient technique. Taguchi grey relational analysis (GRA) Experiment design(11) For the design of the cold plate, multi-objective optimization utilizing Taguchi-gray relational analysis was undertaken, taking into account maximum temperature, temperature standard deviation, and pressure drop The characteristics to be investigated were channel number, channel height, and mass flow rate, with three different levels chosen for each (12).
3. Proposed method
The following components make up the multi-agent concept used in the proposed interconnected system, where each area has its own speed converter (agent).
3.1. Environment
Every time a time step takes place, an agent decides the optimal course of action using the condition of the environment as information, and the environment replies by rewarding that action and generating a new state.
3.2. Observations
The state or observations serve as a representation of the frequency response that the TD3 algorithm, policy, and reward function will use.
3.3. Rewards
As a result, in order to achieve the function of objectivity, the agent is directly motivated by the reward function to take behaviors that maximize values.
3.4. Actions
The policy establishes its significance in order to maximize reward under certain circumstances. The agent continuously explores different PID settings for communication therewith the strength of the network by considering its reward mechanism and response times until it achieves the predetermined goals. The reward function is vital to the solution of the specified load frequency regulation challenge in order to obtain the right PID parameters since it affects the Bellman equation, which is the basis of the suggested TD3 technique Figure 1.
Figure 1. Workflow of the proposed approach.
3.5. Design of controller
The agent is made up of critic and actor networks, where the critic is an estimated value function and the actor is a policy structure that determines what should be done. Player emulates a neural network’s function as the controller for PID, with the fully connected layer serving as the controller output and the feature-input layer receiving inputs that are inverse, essential, and gradient of the ACE. Table 1 lists the variables that were considered for developing character and critic systems for the TD3 agent.
Table 1. Hyperparameters of the proposed converter.
Combining elements are built to connect the two components, and then completely connected layers are added for each input. The Rectified Linear Unit (ReLU) is the activation function for each fully connected layer. To increase the stability of the optimization, the agent constantly updates the targeted protagonist and target critics parameters:
Construct criticism and operator functions in step 1.
Step 2: Define the agent options, such as the mini-batch size, the distribution of Gaussian noise, and the feeling of replay storage length.
Step 3: Create the TD3 agents for both locations based on the characteristics that were supplied in steps 1 and 2.
Step 4: The following algorithm is used for teaching the TD3 agent.
The equal, necessary, and derivation outputs of the PID controller used for training are determined using the absolute weights of the actor-network after retraining is finished. A streamlined version of the PID tuning paradigm is shown in the flowchart in Figure 2.
Figure 2. Flowchart of PID tuning using proposed TD3.
Utilizing an analysis of sensitivity is the easiest method for incorporating uncertainties. An analysis of sensitivity examines the effects of changing each input parameter of the model individually on the outcome. This analysis allows for the identification of the importance of factor fluctuation on the outcome. This approach has been used to examine the effects on financial variables in a DGS system and to reduce the variability of climatic information and load.
This technique has been applied to determine the possibility for demand response and to describe how unpredictability affects the best delivery. However, most of the situations taken into account in susceptibility and what-if evaluations remain random, which results in a poor understanding of the SoU function.
Figures 3(A, B) shows the total number of articles that consider each SOU in various places.
Figure 3. (A) The number of articles that take into account each SoU. (B) The percentage of each SoU that was analyzed in the pool of articles.
4. Experimental results and discussion
In the Taguchi analysis, the control variables and their values should be originally established. Hydraulic power is produced by increasing the gravity energy storage system’s effectiveness. Six control variables that describe the design characteristics are developed using Taguchi analysis to maximize the stored energy. It is created an orthogonal array of L25. The various trials are executed by the simulation model, which then yields the corresponding system output.
Traditional measurements are thought to be less useful as a comparison tool than MSD. S/N ratios can be calculated using one of three methods: greater is more effectively, higher was more effectively or smaller is good Figure 4.
Figure 4. Signal-to-noise ratio on each factor.
Since we initialized the model with [0 0 0] as PID gains, the error penalty for the first episodes was high negative rewards. Due to the presence of RES, Area-2 is subject to more severe penalties. To maximize the reward, the agent chooses the best PID increases as it governs operations. The simulation starts to perform significantly around 100 trials, but it takes 800 trials for the system to improve results and converge on an optimum solution, as shown in the picture. Following model training, the proposed scheme’s robustness is examined in light of various scenarios, and the outcomes are contrasted with those obtained using traditional meta-heuristic and DRL techniques Figure 5.
Figure 5. Moving average of rewards while training.
Figure 6 shows the power of the EV vehicle. Area 1 is initially subjected to a 1% step level disturbance (SLP) in order to evaluate the effectiveness of the control. Figure 6’s results, which show the system’s under-shoot (US) and over-shoot (OS) frequency responses to be 0.006 and 0.0011 Hz, respectively, clearly demonstrate the superiority of the suggested TD3 technique.
Figure 6. Tie-line Power deviation with random SLP.
5. Conclusion
The EMS approach used determines the effectiveness and performance of FCHEV. To improve the performance and economic viability of FCHEVs, it is crucial to design and apply the best EMS approaches. This study examined the most recent power train infrastructure and outlined its benefits and drawbacks. The development of the offline and online EMSs for the FCHEV application was then thoroughly analyzed. The functionality of various EMSs was examined, along with their effects on the FCHEV’s efficiency with hydrogen fuel and SOC upkeep. However, the majority of recent research studies lacked experimental hardware implementation and primarily considered simulation-based analysis. Therefore, such EMSs must be created to provide the best simulation performance and produce acceptable results in real-time applications. The pressure drop with the greatest variance in connection to other result indicators has the largest weighting factor in the Taguchi-based gray relational analysis. The Taguchi orthogonal array-based experiment design was used to decrease the number of experiments required. The data show that boosting the charging current in CC-CV charging results in quick charging; nevertheless, this causes an increase in surface temperature, a loss in charging efficiency, and an increase in heat production.
References
1. Nicolo D, Aruna S, Polak JW. Electric car charging preferences: modelling and implications for smart charging services. Transp Res Part C. (2017) 81:36–56.
2. Wolbertus R, Jansen S, Kroesen M. Stakeholders’ perspectives on future electric vehicle charging infrastructure developments. Futures. (2019) 123:102610.
3. Yin WJ, Ming Z, Wen T. Application of a novel multi-objective optimisation method for EV scheduling in a smart grid under uncertainty. J Ambient Intell Human Comput. (2019) 232:121118.
4. Zhao Y, Che Y, Wang D, Liu H, Shi K, Yu D. An optimal domestic electric vehicle charging strategy for reducing network gearbox loss while taking seasonal factors into consideration. Appl Sci. (2018) 8:191.
5. Amjad M, Ahmad A, Rehmani MH, Umer T. A review of EV charging: From the standpoint of energy optimisation. optimisation approaches and charging techniques. Transp Res Part D. (2018) 2018:386–417.
6. Geetha A, Subramani C. Energy management in hybridization of energy sources for transportation applications is a proposal for student project work. Int J Electr Eng Educ. (2020) 57:253–71.
7. Sankarkumar RS, Natarajan R. Energy management strategies and topologies suited for hybrid energy storage system powered electric vehicles: an overview. IEEE. (2021) 31:1–30.
8. Mohammed O, Eldeeb H, Samy F. Multi-objective optimisation approach for grid-connected PV-powered EV charging station operation. Electric Power Syst Res. (2018) 164:201–11.
9. Han X, Wei Z, Hong Z, Zhao S. Markov chain-based ordered charge control for electric cars with unknown charging loads. Renew Ener. (2020) 161:419–34.
10. Amin A, Tareen WU, Usman M, Ali H, Bari I, Horan B, et al. A review of optimal charging strategy for electric vehicles under dynamic pricing schemes in the distribution charging network. Sustainability. (2020) 12:1–28.
11. Kumar K, Kartik K. Fast charging of lithium-ion battery using multistage charging and optimization with Grey relational analysis. J Energy Storage. (2023) 68:107704.
12. Muhsin K, Gamsz S, Alnca ZN. Comparative evaluation and multi-objective optimization of cold plate designed for the lithium-ion battery pack of an electrical pickup by using Taguchi-grey relational analysis. Sustainability. (2023) 15:12391.
© The Author(s). 2024 Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.