Sunday, November 13, 2016

PowerShell v3 Creating Subfolders with Controlled File Folder Counts

,
Lately I have been doing a lot of basic data manipulation tasks. Basic stuff, but, it has given me a chance to perfect some of my tasks. This post is about how to create folders based on file count in a sequential fashion. So, heres the scenario. Lets say you are me, a document imaging specialist (whatever that means), who has a folder (yes, one folder) with 256, 215 files. Yes, 256k+ files. A lot. Anyone who has tried to work with that many files in either PowerShell or Explorer know that once you get past about 1,000 files in a directory things get a little funky. To make my life simpler I decide to split up this ridiculous file collection into a more manageable set of folders each containing 1,000 files.


Before I take a change with live data I decide to create a small sample set for testing. In my test I create 100 .txt files in a folder. My goal with the script is to move files, 10 at a time, into a numerically sequential folders. In plain English, I want folder 1 to contain files 1.txt through 10.txt. I want folder 2 to contain files 11.txt through 20.txt. Et cetera. In designing this I know, with larger folder sets, that I really want folder names to be zero-filled. Zero-filling is when you pad the left side (in English) of a string with 0s. So, instead of folder name 1, I have can have folder 001. This way when you sort, you get a cleaner folder. Sometimes, with PowerShell, sorting on file or folder name will yield funky results, so, by equalizing all the folder names with zero-filling padded names I eliminte this issue. Yet, for giggles, I touch on a way to maximize sort with expeessions. Enough babbling. Some code.
$path = C:dataDocumentsPowershell est reakingsets
1..100 |
% {
      1 > "$path$_.txt"
}
$counter = 1;
$foldercounter = 1;
$foldersize = 1000;
dir $path |
sort @{e={$_.basename -as [Int]}} |
%{
      # set folder name with zero filling for sorting
      $foldername = ("$path{0:0000}" -f $foldercounter)
     
      # if folder doesnt exist create
      if(!(Test-Path -Path $foldername))
      {
            $folderpath = md $foldername
      }
     
      # Check to see if file name
      if(($counter % $foldersize) -eq 0)
      {
            move $_.fullname $folderpath
            $foldercounter++
      }
      # if file doesnt
      elseif(($counter % $foldersize) -ne 0)
      {
            move $_.fullname $folderpath
      }
     
      # Increment counter
      $counter++       
}
Heres the break down of this script:

  1. Line 1: specify my directory
  2. Line 2: enumerate an array of 100 integers (1..100)
  3. Lines 3-5: create 100 new files named .txt each containing the character 1
  4. Line 6: set a $counter variable to 1 to keep track of file count
  5. Line 7: set a $foldercounter variable to 1 to help increment folder names
  6. Line 8: set a $foldersize variable to 10 to tell the script how many files to put in each directory
  7. Line 9: iterate the contents of my folder
  8. Line 10: sort the contents based on the file basename (that is without an extension) as if they were int objects. This gives you a different sort order than sorting on basename as strings.
  9. Line 13: join the $path and $foldercounter variables into one string (acting as a folder path). The neat thing here is using the format operation -f to zero-fill the folder name. So, when it runs it creates a folder named 0001 instead of 1. This helps when you get to folder 109 which is named 0109 instead.
  10. lines 16-19: create the folder in case it does not exist.
  11. line 22: test to see if the $counter variable is 0 when parsed with a modulus operator equal to the $foldersize variable. The idea here is to increment the folder name by 1 so the next time the loop iterates the script recognizes that the next folder does not exist and to create it.
  12. lines 24-25: move the folder to the new folder and increment the $foldercounter variable.
  13. lines 28-31: same thing as above except it does not increment the $foldercounter variable for the next iteration.
  14. line 34: increment the $counter variable one for each file that passes through the loop.
Okay, a lot to explain a simply task...I know. Nonetheless, there are a few mechanics that, if not unpacked, may not be easily spotted by folks new to this kind of task. The only thing you need to change here is the $foldersize. In fact, you could very easily create a function that looks like this (which I did for reference since this is VERY reusable code):
$basepath = C:dataDocumentsPowershell est reakingsets

1..100 |
% {
      1 > "$basepath$_.txt"
}
     
function Create-SequentialFolders
{
      param(
            [ValidateScript({Test-Path -Path $_})]
            [Alias(Folder)]
            $path,
           
            [Int]
            $counter = 1,
           
            [Int]
            $foldercounter = 1,
           
            [Int]
            $foldersize = 100,
           
            [Int]
            $paddingcount = 3
      )
     
      $path
      $zerofill = 0 * $paddingcount
      Get-ChildItem -Path $path |
      Where-Object {!$_.PSIsContainer} |
      Sort-Object @{e={$_.basename -as [Int]}} |
      %{
            # set folder name with zero filling for sorting
            $foldername = ("$path{0:$zerofill}" -f $foldercounter)
            $foldername
            # if folder doesnt exist create
            if(!(Test-Path -Path $foldername))
            {
                  $folderpath =

0 comments to “PowerShell v3 Creating Subfolders with Controlled File Folder Counts”

Post a Comment

 

Computer Info Copyright © 2016 -- Powered by Blogger