Thursday, July 13, 2017

Speed up inserts for Pentaho on MySQL

I work with Pentaho data integration a fair amount at work.  When I try to run jobs locally the insert speed is ridiculously slow.  I found this article that gives a good way to speed up the inserts.

TLDR: add useServerPrepStmts=false, rewriteBatchedStatements=true and useCompression=true to database connection options and things speed up quite a bit on the insert side.  I still need to find a way to may insert/update faster.

No comments:

Post a Comment