Parallel connections to DB

[Michlus]

Help #2:
As I mentioned in my previous post, I run 2 scripts with PQ on SQL DB.
But actually I send each of those scripts to run on 4 DBs. The data is then appended in 2 PQ tables.
They're heavy scripts. Can take 10 minutes each.
Is it possible to engage 4 connections simultaneously (script1 on the 4 DBs) and then script2 on them?
Parallel running will complete in some 20 minutes.
Sequence running can take 10*2*4 = 80 minutes.
Help is appreciated.
Thanks,
Michel

 

[Dan Bliss]

Don't see link to previous posts.. Provide Links?

I'm wondering if either of these might work to run queries simultaneously for you?

Source1 = ...
Source2 = ...
let

...= [A PQ function](Source1)
...
...= [A function](Source2)

​

or

let

Source1 = ...
let​

Source2 = ...​

on​

Script2​

on

Script1

​

If possible a good strategy is to tease out from your PQ scripts any heavy filtering and run in DB as a SQL statement, cutting down the number of rows presented to PQ as source.
I assume your scripts have not thwarted "query folding"?

* Don't use SQL statements as a connection to a source (not the same as described above),
* Connect to tables, not views,
* Build keys in PQ for columns you are filtering or sorting,
* Lay off use of dynamic parameters,
* Use UI rather than custom M code where possible.

Performance improves when connected to SQL-Server.

Hope this helps.