Sunday, November 13, 2016

PowerShell v3 Creating Subfolders with Controlled File Folder Counts

,
Lately I have been doing a lot of basic data manipulation tasks. Basic stuff, but, it has given me a chance to perfect some of my tasks. This post is about how to create folders based on file count in a sequential fashion. So, heres the scenario. Lets say you are me, a document imaging specialist (whatever that means), who has a folder (yes, one folder) with 256, 215 files. Yes, 256k+ files. A lot. Anyone who has tried to work with that many files in either PowerShell or Explorer know that once you get past about 1,000 files in a directory things get a little funky. To make my life simpler I decide to split up this ridiculous file collection into a more manageable set of folders each containing 1,000 files.


Before I take a change with live data I decide to create a small sample set for testing. In my test I create 100 .txt files in a folder. My goal with the script is to move files, 10 at a time, into a numerically sequential folders. In plain English, I want folder 1 to contain files 1.txt through 10.txt. I want folder 2 to contain files 11.txt through 20.txt. Et cetera. In designing this I know, with larger folder sets, that I really want folder names to be zero-filled. Zero-filling is when you pad the left side (in English) of a string with 0s. So, instead of folder name 1, I have can have folder 001. This way when you sort, you get a cleaner folder. Sometimes, with PowerShell, sorting on file or folder name will yield funky results, so, by equalizing all the folder names with zero-filling padded names I eliminte this issue. Yet, for giggles, I touch on a way to maximize sort with expeessions. Enough babbling. Some code.
$path = C:dataDocumentsPowershell est reakingsets
1..100 |
% {
      1 > "$path$_.txt"
}
$counter = 1;
$foldercounter = 1;
$foldersize = 1000;
dir $path |
sort @{e={$_.basename -as [Int]}} |
%{
      # set folder name with zero filling for sorting
      $foldername = ("$path{0:0000}" -f $foldercounter)
     
      # if folder doesnt exist create
      if(!(Test-Path -Path $foldername))
      {
            $folderpath = md $foldername
      }
     
      # Check to see if file name
      if(($counter % $foldersize) -eq 0)
      {
            move $_.fullname $folderpath
            $foldercounter++
      }
      # if file doesnt
      elseif(($counter % $foldersize) -ne 0)
      {
            move $_.fullname $folderpath
      }
     
      # Increment counter
      $counter++       
}
Heres the break down of this script:

  1. Line 1: specify my directory
  2. Line 2: enumerate an array of 100 integers (1..100)
  3. Lines 3-5: create 100 new files named .txt each containing the character 1
  4. Line 6: set a $counter variable to 1 to keep track of file count
  5. Line 7: set a $foldercounter variable to 1 to help increment folder names
  6. Line 8: set a $foldersize variable to 10 to tell the script how many files to put in each directory
  7. Line 9: iterate the contents of my folder
  8. Line 10: sort the contents based on the file basename (that is without an extension) as if they were int objects. This gives you a different sort order than sorting on basename as strings.
  9. Line 13: join the $path and $foldercounter variables into one string (acting as a folder path). The neat thing here is using the format operation -f to zero-fill the folder name. So, when it runs it creates a folder named 0001 instead of 1. This helps when you get to folder 109 which is named 0109 instead.
  10. lines 16-19: create the folder in case it does not exist.
  11. line 22: test to see if the
$counter variable is 0 when parsed with a modulus operator equal to the $foldersize variable. The idea here is to increment the folder name by 1 so the next time the loop iterates the script recognizes that the next folder does not exist and to create it.
  • lines 24-25: move the folder to the new folder and increment the $foldercounter variable.
  • lines 28-31: same thing as above except it does not increment the $foldercounter variable for the next iteration.
  • line 34: increment the $counter variable one for each file that passes through the loop.
  • Okay, a lot to explain a simply task...I know. Nonetheless, there are a few mechanics that, if not unpacked, may not be easily spotted by folks new to this kind of task. The only thing you need to change here is the $foldersize. In fact, you could very easily create a function that looks like this (which I did for reference since this is VERY reusable code):
    $basepath = C:dataDocumentsPowershell est reakingsets

    1..100 |
    % {
          1 > "$basepath$_.txt"
    }
         
    function Create-SequentialFolders
    {
          param(
                [ValidateScript({Test-Path -Path $_})]
                [Alias(Folder)]
                $path,
               
                [Int]
                $counter = 1,
               
                [Int]
                $foldercounter = 1,
               
                [Int]
                $foldersize = 100,
               
                [Int]
                $paddingcount = 3
          )
         
          $path
          $zerofill = 0 * $paddingcount
          Get-ChildItem -Path $path |
          Where-Object {!$_.PSIsContainer} |
          Sort-Object @{e={$_.basename -as [Int]}} |
          %{
                # set folder name with zero filling for sorting
                $foldername = ("$path{0:$zerofill}" -f $foldercounter)
                $foldername
                # if folder doesnt exist create
                if(!(Test-Path -Path $foldername))
                {
                      $folderpath =

    0 comments to “PowerShell v3 Creating Subfolders with Controlled File Folder Counts”

    Post a Comment

     

    Computer Info Copyright © 2016 -- Powered by Blogger