The design ICT systems that can provide application services that quickly and flexibly integrate network and software is currently the key method in DX based on ICT. Non-etheless, the problem of the huge amount of time required for the automatic design of ICT systems has been paid more and more attention. Considering the rapidly changing demand for ICT systems in various industries, the need to build and monitor systems frequently, and the difficulty in securing engineers due to the declining birthrate and aging population, the huge design time problem cannot be ignored, especially if AI is expected to reduce design time. To this end, we study the problem of reducing the design time of ICT systems with deep reinforcement learning algorithms , Weaver. First of all, we set the graph neural network as the learning model, so that deep reinforcement learning algorithm is used to solve this problem. Secondly, we have designed an algorithm that combines the representative reinforcement learning algorithm Double DQN and Noisy Network. Then, we attempted to shorten the design time by changing the normal distribution to a truncated normal distribution at the optimal value among them. Finally, sufficient trials are conducted to verify our proposed method. The results show that, the proposed method reduces the learning time required to complete learning of the design.