Air conditioning must be provided where it is needed and the ideal cooling arrangement usually involves the careful management of cool and warm air flows. “Enterprises must … carefully factor in the power and cooling demands of these blades,” says a recent Gartner Group report. “Many organizations will not be able to achieve optimum density because of environmental limitations.” ASHRAE’s Thermal Guidelines for Data Processing Environments (2004) says that temperatures should be maintained between 20 degrees Celsius and 25 degrees Celsius in the server room. As temperatures rise, failure becomes a real possibility, which can lead to downtime and even data loss.
The challenge was especially great in a recent project where the computer room air conditioner (CRAC) units were located in mechanical rooms on either side of the server area. This approach makes it possible to keep the server area locked and sealed even while the CRAC units are being serviced. The room was originally designed so that cold air would flow out of the CRAC units, under the raised floor, and up through perforated tiles into the server area, forming cold aisles between racks of servers. The cold air would then be drawn into the air intakes of the server racks, where it would take on heat from the equipment. The hot air would then exit from the backs of the servers into hot aisles, where it would be drawn up to the ceiling and through perforated panels for return to the CRACs.
One approach to estimating cooling requirements for data centers involves the use of hand calculations to estimate the performance of various alternatives. The problem with these calculations is that they require many simplifying assumptions, thus limiting their accuracy in practical applications. Another approach is the use of energy analysis programs that accept as input a description of the building layout, systems, construction, usage, and utility rates, along with weather data. The limitation of this type of program is that is primarily intended to evaluate energy usage and cost and thus can only predict average temperatures for a particular space. Of course, knowing that the average temperature in the data center is 22 degrees Celsius would be small comfort if the air temperature near one critical server is 29 degrees Celsius.
In the past, about all that engineers could do was to make sure the average temperatures were right and hope that there wasn’t too much variation in the data center. In this building, because of the unique design, there was a real risk that after the building was finished temperatures in the data center would be too high. It would then be necessary to go through a lengthy and expensive trial and error process in order to remedy the problem.
Facilities Engineering Associates (FEA) felt that computer simulation was essential to resolve the unique nature of the cooling problems in this application. CFD can provide enormous assistance by calculating and graphically illustrating the complete airflow patterns, including velocities and distributions of variables such as pressure and temperature. They selected Fluent Incorporated, Lebanon, NH, as consultants to perform the analysis because of Fluent’s leading position in the CFD industry and its experience in addressing heating, ventilation, and air conditioning applications. Their goal was to identify potential problems in the room prior to construction, in order to prevent them from occurring later on, which would require costly downtime for after-the-fact modifications. The process of simulating airflow in a data center has been greatly simplified by the development of Airpak CFD software from Fluent Inc., because it is designed specifically for modeling internal building flows.
Pathlines colored by temperature for the perpendicular rack case, simulated as part of the proof-of-concept phase of the project
Pathlines colored by temperature for the parallel rack proof-of-concept case
Simulating the proof-of-concept designs
The Fluent consultant assigned to the project began by modeling the geometry of the two proposed proof-of-concept designs. The main goal of this analysis was to evaluate the placement of the high density rack relative to the position of the CRAC units in the data center room. In one proposed layout, the CRAC units are parallel to the server racks and in the other design, they are perpendicular. The server and mechanical rooms were defined as boxes, and the raised floor, physical rack locations, CRAC units, suspended ceilings, beams, and tiles were created for each case. One server rack was assigned a higher power density than the others, and the impact of its presence and location in the room for each case was studied. The simulation results provided the air velocities, pressures, and temperatures throughout the server room and lower plenum, and were used to illustrate the resulting air flow patterns for each layout. The global air flow patterns showed that the design worked as intended, with the air rising through the perforated sections of the floor, flowing through the racks, and exiting through the returns in the ceiling.
Zoomed view of pathlines in the vicinity of high-density rack for the perpendicular proof-of-concept case
While the design seemed to be working well at first glance, closer examination showed several serious problems. The figure above shows the flow in the area of the high-density rack, which is positioned next to the wall in the perpendicular orientation case. Cooling air is drawn into the cold aisle between this rack and the adjacent low-density rack through the perforated floor tiles. Following the temperature-coded pathlines, it can be seen that the hot air exiting from the high-density rack circles over the top of the rack and re-enters the cold aisle, where it is drawn into the upper intake portion of the neighboring low-density rack. The lower portion of the low-density rack is not affected. It continues to be properly cooled by air entering through the floor tiles. Temperature contours on the rack intakes (see figure below) further illustrate the problem. A similar problem occurs for the parallel orientation case, where the high density rack is positioned between two normal density racks. Hot air exiting from the high-density rack circles above and around the sides of the rack and re-enters the cold aisle, compromising the cooling air that is delivered through the floor.
Surface temperatures on the rack inlets in the cold aisles for the perpendicular proof-of-concept case
Evening out the heat load
The proof-of-concept simulations demonstrated that the root cause of the problem was with the layout of the high-density racks. It was apparent that the original design concentrated the high load racks in a single row, which created a hot spot in that area. Working within the constraints provided by the client, engineers repositioned the servers to even out the heat load throughout the room. The engineers also addressed the fact that the heat load of the different racks varies widely, from 1 to 7 kilowatts. The original model used the power ratings provided by the blade manufacturers, but the engineers recognized that deviations from the rated values could greatly impact the accuracy of the simulations. They therefore measured the power draw of each unit and discovered that the actual values were considerably less than the rated ones. Using this information, along with information from the simulations, the engineers were able to determine the air conditioning capacity required to cool the room. The calculation was based on the requirement that the failure of any single CRAC unit would not cause the temperature to rise to dangerous levels. Finally, they examined the predicted pressure losses throughout the room and lower plenum to determine the requirements of the fans used to drive the system.
FEA and Fluent engineers also evaluated the effect of the plenum height on the flow patterns in this region. The results showed that a jet of air traveled through the 10.5 inch high plenum and struck the opposing wall, causing a recirculation pattern with nonuniform flow conditions and localized high pressure zones. Assuming that the height of the plenum was the cause of the problem, engineers generated several different models with a plenum of various heights. They discovered that a plenum height of approximately 16.5 inches was the smallest that would promote smooth air flow into the server room. In order to straighten out the airflow through the perforated floor tiles, engineers specified ductwork with an aerodynamic shape that creates a uniform static pressure across the entire space and also absorbs noise.
This application demonstrates that CFD can be used to resolve problematic areas within existing or planned data centers. By using CFD in the design process, potential cooling problems, such as those illustrated in this example, can be identified before they occur, saving the time and expense required for repairs and retrofitting, once the center is on-line.
This comment has been removed by the author.
ReplyDelete