PURPOSE OPERATION COMMAND LINES OPTIONS RELATED PROGRAMS
Author: Dan Mares, dmares @ maresware . com (you will be asked for e-mail address confirmation)
Portions Copyright © 1998-2021 by Dan Mares and Mares and Company, LLC
Phone: 678-427-3275
One liner: finds alternate data streams of files and extracts to "normal" file
This is a command line program.
MUST be run within a command window as administrator.
NOTE: This program has NOT been updated to be able to handle the long > 255 character path/filename lengths. To process files with long filenames, check out the UPCOPY program with either or both of the --ADSPARENT or --ADSCHILD options. These two options in upcopy will process the Alternate Data Streams for long filenames.
The Copy_ads program has three main purposes. But lots of auxiliary uses.
The first is to traverse a drive or tree structure of an NTFS file system and identify all the NTFS Alternate Data Streams on the drive. This is the default operation. However, with newer versions of Windows, there are so many alternate data streams created as a matter of course, this operation has become almost irrelevant.
The second purpose is to copy any files containing Alternate Data Streams plus the Alternate Data Streams themselves to a designated destination directory. The Alternate Data Streams will be extracted and placed in normal files so additional processing of the alternate data stream (which is now a normal file) can be accomplished.
On March 3, 2011, the third option was added. This option is to extract or copy ONLY the alternate data stream file and place it in the destination directory (-d destination_folder option). This option was added as an enhancement to the hash programs' --ADDADS option( which performs an MD5 or SHA1 hash of a file, and adds an alternate data stream to the file with appropriate meta-data. See operation and options section below for more information.
The "engine" which locates and creates the destination directories is similar to the Upcopy program. Users should consult the Upcopy documentation to see how the destination paths (directories) are created.
Copy_ads is designed to read a source and destination directory and copy files and their Alternate Data Streams from the source (-p option) to destination (-d option). It operates in a similar fashion to the Maresware Upcopy program but is designed to work on the Alternate Data Streams files.
Generally it should be used to identify and/or copy Alternate Data Stream files from one directory to another.
During the copy process the file dates and times of the original file are maintained in the destination file attributes. However, without the -R (Reset) option, the original access time of the source file is modified accordingly.
The simplest and more basic of the operations is the program's capability of identifying Alternate Data Streams on an NTFS file system. The program searches the source path, (the default path, or -p option, or for shorthand, dr:\path\..\.. without the -p) for files meeting any additional option criteria provided on the command line. The default is to search all files in the provided source (default) path.
If the file meets the command line option criteria (see options below), it is then (and only then) checked to see if an Alternate Data Stream (ADS) file is associated with the file. If so, information regarding the file and the ADS is printed on the screen. The information displayed is: filename; size; date/time. Similar information for the ADS is also provided. The suggested mode of operation is to allow the program to search all files.
The output is displayed on the screen, and may be somewhat confusing and voluminous. For this reason it is suggested that the -o (output) option be used to capture the listings of the identified files. If the -o output option is used, then this information is placed in an appropriate output file.
WIN2K (NTFS5) has the capability of adding "file summary information" to files. This is done by right clicking a filename in Explorer, and then going to the properties tab and selecting the summary information tab. These statistics are maintained (currently) in at least 3 ADS's. One of these has a name of over 250 characters, and is mostly filled with blanks. For this reason, the output line is very long and can be reduced significantly by using the -w option.
This ADS identification process can be run as a stand alone and will not copy or alter any information in the file or ADS.
NOTE: When talking about "files" in the search and copy process here we are generally referring to those files with Alternate Data Streams.
The user provides a starting path or directory for the program to start at. This starting point can be the default directory. Or, if the -p option is used, the starting path can be any directory identified with the -p option.
The user also provides the file types to check for Alternate Data Streams. The default file type is *.* (all files). With the use of the -f option, the user can restrict the analysis to files meeting the file type specified. (ex., *.exe ).
The user provides as a required option, a destination "TOP" level directory for all identified files to be copied to. See Destination Path below for explanation of how the final destination is determined.
To start, the program locates all files in the source path(s) meeting the file specifications (-f *.*). If no source path is provided, the default path is assumed (modified by the -p option). If no file types are provided (-f option) then the *.* is default and is assumed. Source file types can contain wildcards, multiple file specs, or be blank for all (*.*)
The recommended path and file types would be all files from a root directory. A source directory (ex., C:\) must be specified, but source file types (-f option) defaults to (*.*).
When the program starts, it searches the source directory for "normal" files meeting the criteria specified on the command line. (There are numerous option combinations. SEE the options section below). When a "normal" file is found meeting the command line criteria, it is examined to see if there is an Alternate Data Stream associated with the file. If an Alternate Data Stream is identified, the source file and all the ADS files are copied to a destination directory.
The "normal" or main file is copied to the appropriate destination directory, just as if the copy command had been issued from the command line.
A technical note about copying the files to another NTFS partition: When the main file is copied from one NTFS system to another, if it contains ADS files, then all those ADS files are copied also. So any main file that is copied will have all its ADSs copied with it. Later on, when searching the destination for ADSs, all those copied ones will be there along with the "extracted" copies.
Because the destination may or may not be an NTFS file system, the ADS files which are part of the main or normal file are copied to the destination as "normal" DOS files and renamed using the following logic or mask:
The original ADS filename: FILENAME.EXT:ADS.AXT is altered in the following manner: the colon (:) in the ADS file is replaced with the left square bracket ( [ ) and the right square bracket ( ] ) is appended to the ADS name. This now provides a legitimate DOS/WINDOWS long filename to use. FILENAME.EXT[ADS.AXT]
The file FILENAME.EXT:ADS.AXT is now copied to the appropriate destination location with a "normal" DOS file name of: FILENAME.EXT[ADS.AXT]. No checks are made to see if the destination already exists, and it is overwritten if it does exist. The user is responsible for providing a destination directory which will be suitable for this copy process.
Once the files are copied they can be treated as normal DOS files, which means the user may examine and manipulate them with normal procedures.
topWhen the copy function takes place, the original source path/directory is replicated below the destination "TOP" level directory (as provided with the -d option). Therefore, the original source/path is always intact and available on the destination drive below the -d option directory. This directory creation process maintains a consistent tree/path from the destination directory provided on the command line down to the final file location. This is useful for later forensic analysis or just to document where the original files were located.
For example:
Source file location: c:\winnt\system32\anyfilewithan_ADS
Destination directory provided on the command line with the -d option. -d D:\ADS_FILES
Final destination directory created to hold the files: D: \ADS_FILES\ winnt\system32 (Notice the winnt\system32 is appended to the -d option D:\ADS_FILES.)
Any files located will now be copied to the D:\ADS_FILES\winnt\system32 directory.
When the same file is found on the destination as in the source directory, and the destination file is older than the source file, the newer source file is copied to the destination. If the file being copied is the Alternate Data Stream file, it is ALWAYS overwritten regardless of age. (The trigger for the date test is ONLY based on the date/time of the normal file, not the ADS file.)
The destination (-d option) MUST be a directory or drive.
Note: If you are using only a drive as destination, (-d d: ) make certain that the default directory on that drive is the top level directory to start writing to. Ex., if the default on A: was A:\tmp when the operation started, then all the copying would begin at A:\tmp and continue to create subdirectories below. If you wanted to start at A:\ then the default directory on A: should be root.
There is no check to see if the destination file is read-only. All destination files are currently overwritten. It is the user's responsibility to make certain no essential files are overwritten during this process.
Various options exist to allow the user to “program” the file selection process by file name, size, age, whether the destination exists or not, and other options.
During the copy process the file dates and times of the original file are maintained in the destination file attributes. However, without the -R (Reset) option, the original access time is adjusted accordingly.
topThe "--REVEAL" Alternate Data Stream operation is the one implemented March 2011. This operation is meant to find the ADS, and extract ONLY the ADS from the file, and place it in the destination directory. It is similar to the default copy operation, but it does NOT copy the main file. It only copies/extracts the Alternate Data Streams. This way, if you create a destination folder and possiblye call it "REVEALED_FILES", what you will end up after this process, is a destination folder with all the data streams showing up in "visible" files, which can be seen and accessed thru the normal explorer window.
This capability is extremely useful when following this proceedure. First you run the hash program with the --ADDADS option. This will create an Alternate Data Stream which contains file meta data (including date and times) for every file which is hashed. Now suppose, you are wanting to provide those files on a CD to a client. The normal CD writing process does not copy any data streams. So all the meta data, which included key file dates and the hash, would be lost in a normal CD writing process. However, if this --REVEAL option was run on the folders containing the files you wish to burn to the CD, then this process would create a seperate directory which would contain the Alternate Data Streams (which would be those created by the hash program), in visible (clear text) files. Then you merely include this new "REVEALED FILES" folder in your CD writing process, and the client not only has the original file, but sufficient meta data (including key file dates) with which they can reference.
On a side note. This process of using hash to create and alternate data stream with the hash and file date/times is nothing more than a sophisticated or alternate way of running hash with a single output file. Then you have a hash catalog and date/times of the files. Which can be kept as reference. The main difference, is that the altnerate data streams will stay with the file if it is copied to another NTFS drive, while a seperate hash output file will have to be managed seperately. (Remember, there is more than one way to skin a cat. But don't mention that to my kitties.)
This is a shortcut command line and should only be used by experienced users. It is preferable to use all options when providing a command line.
C:>COPY_ADS [source_directory] [-[options]]
C:>COPY_ADS [source_directory[c:\]] -o outputfile
/* locate and print to outputfile all files with ADS */
C:>COPY_ADS [source_directory[c:\winnt]] -o outputfile -v
/* locate and print to outputfile all files with ADS, eliminate the headers/footers in outputfile */
C:>COPY_ADS [source_directory] -o outputfile -r
/* locate and print to outputfile all files with ADS, DO NOT recurs tree to find files. */
C:>COPY_ADS C:\tmp -d D:\tmp\old
/* copy the tree structure from C:\tmp to a new directory D:\tmp\old */
C:>COPY_ADS -p C:\tmp -d d:\tmp\old
/* same as the first one, except this one makes use of the -p option also */
C:>COPY_ADS -p c:\tmp -d d:\tmp\old -f *.doc
/* copy only the *.doc files*/
C:>COPY_ADS -p c:\tmp -d d:\tmp\old -f *.doc *.ppt
/* copy *.doc and *.ppt files *
C:>COPY_ADS -d d:\work_dir -S listfile.nam
/* copy all the files identified in the text file listfile.nam to the d:\work_dir tree */
C:>COPY_ADS -p c:\ -d d:\work_dir -N
/* when copying the files to another NTFS location, the main file data streams are not copied.
Only the data from the main file is copied. The Data Streams are still copied to their own
individual data files. See -N option. */
C:>COPY_ADS -p c:\files_which_have_the_hash--ADS_option -d d:\revealed_files
--REVEAL
/* copy or extract ONLY the alternate data streams from those files which
have them. Place the "revealed" files in the destination directory as visible files.*/
Format note: When an option is listed of the following format: -option + filename: the plus sign (+) means that you must include an item following the option. DO NOT INCLUDE the plus (+) in the command line.
-p + src_dir Use this directory as the source (starting point). (-p c:\winnt\system32)
-d + dest_dir Use this as the destination directory. This is the top level destination path. All files will be created under this destination, and the original path will be maintained below this destination. If no -d option is used, then the copy process is not installed, and ONLY a listing of the ADS files will be provided. It is suggested that in any case, the -o option is also used for record keeping purposes. ( -d d:\work\casename)
-f + filetype(s) Copy only those files meeting this file type. Additional file type can be added by separating each one by a space. (ex., -f *.c *.doc *.tmp *.ppt ).
-x + filetype(s) The -x (eXclude) these file types, will exclude files meeting these name requirements from the search. This function is similar in operation to other Maresware programs using the eXclude option. This option and the -f option only trigger on the normal filename and don't examine the ADS filename. (-x *.com *.dll )
-N: The -N option is for 'N'o copy of the Alternate Data Streams with the main file. Normally, when copying the main file to another NTFS destination, the main file data is copied in a normal fashion. This means that ALL the Alternate Data Streams associated with the file, are copied with it also and remain as data streams. Just as if you copied the file using the copy command. Some individuals might want to isolate completely the main file, and not have any data streams copied with it. The usual way to do this is to copy the file to a FAT partition. This eliminates the ADS. But many people don't have a FAT partition to copy to. So this option will copy only the main file portion to the destination, and not the data stream section. This is also, a round about way to eliminate data streams from files. NOTE: this does not affect the extraction and writing of the individual data streams to normal files. (Which is the purpose of this program).
--REVEAL: (March 2011) The --REVEAL option is almost an exact opposite of the -N option. The --REVEAL option, copys or "REVEALS" ONLY the Alternate Data Stream file, and does NOT copy the parent or main file. This places ONLY the data streams in the destination directory. It is useful to run this option on folders after the hash program option --ADS has been run. It effectively copyies out the ADS files to visible files. This option is also useful if the parent file is very large and there is really no reason to copy the larger parent file when you are only looking to reveal the ADS files. DO NOT MAKE THE DESTINATION -d directory THE SAME AS THE SOURCE (-p ) DIRECTORY. UNKNOWN RESULTS WILL OCCUR AND YOU MAY SEND THE PROGRAM INTO A LOOP because the program will continue to process files which it is currently creating.
-o + outputfilename The outputfilename is used when a listing of all the Alternate Data Stream files are needed. This option is independent of the copy options. So the -o can be used to merely provide a list of files and the ADS. (-o d:\work\ads_catalog )
-a If the outputfilename (-o) already exists, append to the existing file.
-v No Verbose. If the -o option is used, a header and footer are normally placed in the output file. This no Verbose option is used if these headers and footers are to be eliminated. This makes for a cleaner data type output file.
-D + delimeter If the -o option is used to create an output file this option will place the delimeter indicated between fields of the output file records. Each output record is fixed in length, and contains the filename, filesize, date/time of the file. The -D option adds an appropriate delimeter for additional reprocessing.
-w + maxpath_Width The max path Width is used if the output records are truncated for long paths. The default output path size is 75 characters. Many directories may be larger. If it is necessary to expand the size of the path in the output records, the -m option is available. (-w 150)
-r DO NOT recurse through the source directory for file. The default is that the source directory is recursed and ALL subsequent files and directories are copied. The default operation emulates the XCOPY command.
-E Create a file called NO_COPY.ERR in the current default directory. (Occasionally the program will place this file in the root of the current default drive.) This file contains a listing of all files not copied to the destination. Two common reasons why the file could not be copied are: (1)the destination disk is full; and (2)the destination file is locked by the OS ( NT locks certain files and they can't be accessed while locked).
-g + #
-l + # Copy only those files
(g)reater than or (l)ess than # days old. Replace the # with a valid
number of days. ( -l 5 )
-t[acw3] This is used to modify the -g or -l option to specify which time type to use in the calculations. The a= =access, c= =create, w= =last write time. Don’t forget, in WIN9X, there is no access time. Or, if the -t3 is provided, then all three file times are added to the output records.
-G + #
-L + # Copy only those files
(g)reater than or (l)ess than # bytes in size. Replace the # with a
valid file size. ( -G 1000000)
-R Because the files are opened and read, on WINNT and WIN9X the access date is modified. This option attempts to reset the source file date back to its original.
During the copy process the file dates and times of the original file are maintained in the destination file attributes. However, without the -R (Reset) option, the original access time is adjusted accordingly.
-1 logfile (that's a one, not an ell) Create an output logfile with statistical information relating to the programs operation. (-1 d: \work\ads_logfile )
-2 logfile Create an output log file with statistical information. This file is more verbose than that used in the -1 logfile option, and contains a list of all the source files and their destination. This file can get quite large depending on the number of files copied.
STREAMS from sysinternals.com
top