[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
how to split the files up into ~equal groups when moving them to other s
From: |
Alex Muir |
Subject: |
how to split the files up into ~equal groups when moving them to other server |
Date: |
Tue, 17 Jul 2012 15:13:28 -0400 |
Hi,
I've setup the following commands, the first to move files to another
server and the second to distribute a process over the 2 computers.
Currently I'm just testing with a virtual box.
My question though is how to split the input files up into ~equal
groups when moving them so that the process is truly distributed such
that each process runs with different input files. Currently what I
have below would make the process run with all the same files on the
local and virtual box server.
Transferring Files
ls /mnt/xslt_volume/i4ContentSource/578/*.zip | parallel --sshlogin
192.168.56.10 --transfer mv *.zip
/mnt/xslt_volume/i4ContentSource/578/
Distributing Work
sudo ls /mnt/xslt_volume/i4ContentSource/578/*.zip | parallel -j-1
--eta --progress --sshlogin 192.168.56.10,: --timeout 1500 "sh
/mnt/xslt_volume/i4EnrichV4/src/enrich/10k/completeMultiDocumentProcess/processExecution/configuration/run-calabash-withParam.sh
/mnt/xslt_volume/i4EnrichV4/src/enrich/10k/completeMultiDocumentProcess/processExecution/configuration/Createi4EnrichMarkup-Template-ConvertedToParameterDrivenFile.xpl
/mnt/xslt_volume/i4EnrichV4/src/enrich/10k/completeMultiDocumentProcess/processExecution/logs/{/.}Log.txt
{}"
Thanks
--
-
Alex G. Muir
Software Engineering Consultant
Linkedin Profile : http://ca.linkedin.com/pub/alex-muir/36/ab7/125
- how to split the files up into ~equal groups when moving them to other server,
Alex Muir <=