PT Study Finds The Dell Poweredge XE9680 Servers With NVIDIA H100 Gpus Supports Many Chatbot Users

(MENAFN- EIN Presswire)

Running your in-house chatbot using Llama 3.1 405B LLMs on Dell PowerEdge XE9680 servers with NVIDIA H100 GPUs

ROUND ROCK, TX, UNITED STATES, May 13, 2025 /EINPresswire / -- Enterprises of all types and sizes are seeking to harness the power of GenAI to meet customer service needs via chatbots that can solve problems without human intervention. Finding a hardware solution that can support the right number of simultaneous chatbot users for a specific use case is crucial to success. Principled Technologies (PT) tested a Dell PowerEdge XE9680 servers with H100 GPUs to show the kind of chatbot performance an organization might expect with this solution.

According to the report, Principled Technologies“used the PTChatterly service to showcase the AI chatbot performance of a DellTM PowerEdge XE9680 server powered by eight NVIDIA H100 SXM TensorCore GPUs, each with 80 GB of memory. For testing, we used a very large LLM (Llama 3.1 405B) and FP8 precision, which requires very fast accelerators and ample memory. With this LLM, which we augmented with in-house data, we quantified the number of supported users engaging in longer conversations to mirror what real-world users would experience. We used sample in-house data in conjunction with the LLM to address organizations that want the precision of a very large LLM using FP8 precision.”

In this testing, PT found that a single Dell PowerEdge XE9680 server with eight NVIDIA GPUs could support 68 simultaneous chatbot users engaging in lengthy questions and answers. The report goes on to note that“for a 60kW rack of six PowerEdge XE9680 servers, we estimate that such a solution could support 408 simultaneous users at a cost of approximately $8.3M over the next five years. If your organization needs the precision of a very large LLM using your own data, the Dell PowerEdge XE9680 server with NVIDIA H100 GPUs can provide ample resources to power your in-house chatbot.”

To learn more about PT testing on the Dell PowerEdge XE9680 server with NVIDIA H100 GPUs, read the full report at .

About Principled Technologies, Inc.
Principled Technologies, Inc. is the leading provider of technology marketing and learning & development services.

Principled Technologies, Inc. is located in Durham, North Carolina, USA. For more information, please visit .

Sharon Horton
Principled Technologies, Inc.
...
Visit us on social media:
LinkedIn
Facebook
YouTube
X

Legal Disclaimer:

EIN Presswire provides this news content "as is" without warranty of any kind. We do not accept any responsibility or liability for the accuracy, content, images, videos, licenses, completeness, legality, or reliability of the information contained in this article. If you have any complaints or copyright issues related to this article, kindly contact the author above.

MENAFN13052025003118003196ID1109543359