Here is my setup
Mgm Node: 1x (r5.xLarge) Data Nodes: 2x (r5.4XLarge) Mysql Nodes: 2x c5n.9XLarge --ndb-cluster-connection-pool=4 --max-connections=16000
Table i am running my KV lookup on: (the table has 150 million rows)
CREATE TABLE `my_table` ( `ID` int NOT NULL, `column1` double DEFAULT NULL, `column2` int DEFAULT NULL, PRIMARY KEY (`ID`), KEY `col2_idx` (`column2`) ) ENGINE=ndbcluster DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_0900_ai_ci
on generating load:
Key lookup requests similar to this:
(select id,column1 from my_table where id = ?)
The load hits and plateaus at: 20Million requests per minute (333k req/sec)
DataNode CPU hits 95-97%%+
Mysql nodes CPU hits 98-99%%
not able to generate any more load due to CPU going this high.
i am running a rest service with 20 nodes which create a connection pool with the mysql instance using Connector/J. Each node creates a pool with mac 30 connections at the moment.
the p90 latency i am getting is ~2ms which is perfect but given the benchmarks i was expecting this to go beyond 2 million req/sec at bare minimum
@mikaelronstrom could this be some sort of a misconfiguration?
i am using the default setup … the only additional params i have are on the mysql side :
any help would be appreciated
i am just looking forward to seeing the benchmark numbers as close as possible
let me know if you need anymore info from my end.