*** *** *** *** *** *** *** *** *** *** *** ***

FAST! File Weeder 2.0 - Copyright (C) 2002 Bumblebee

        Contents:

        1. Introduction
        2. Features
        3. Usage
        4. History
        5. Known Issues

*** *** *** *** *** *** *** *** *** *** *** ***


        1. Introduction

        This is a file weeder using MD5 (RFC1321). It's intended to be fast
and low memory consuming. For this purpose it is optimized for win32 systems
(if you're very interested about porting it to another 32 bits systems lemme
know). If you wanna send me a bug report or whatever, try to find someone
that can find me.

        This program is freeware and is provided "as is". Any kind of implied
or express warranties are disclaimed. This program is intended to be used,
so feel free to redistribute it in any way.

        I coded this weeder as challenge between me and my good friend
VirusBuster. Many thanks to him for his support and help while developing
this little tool.

        At current point of the development, fweeder is one of the fastest
file weeders using MD5 under win32 systems.

        VS2000 distribution is the prefered place to get FAST! File Weeder
releases, but since version 1.2 you can get them also from simtel.net.


        2. Features

o Intended to be fast al low memory consuming.

o It uses MD5 as secure hash function.

o Has common functions required for a file weeder (create database, add
  files to existing database, check for duplicate files aganist a database,
  manage different databases, delete duplicated files, ...).

o Stadistical reports.

o It's a small and lightweight win32 console application.

o Support for long filenames (including spaces in path, just use "s in the
  command line).

Other interesting things

o VirusBuster and me try to make databases compatible (since VirWeeder Plus
  version 1.2, next may follow).

o Easy to work with several databases.

o CRC32 + file size as alternative 'secure' hash.


        3. Usage

  fweeder <command> [<switch>]

  available command list:

  -h                Little help screen

  -c <path>         Create a database for this path.

                    It will dump a report too with the names of the
                    duplicated files in a quite standard format:

                    FILE_DUPLICATE is a duplicate of FILE_ORIGINAL

                    as well as some stadistical data.

  -a <path>         Add files to current database.

                    Old database is saved with .old extension. A report
                    will be created with duplicated files in the same
                    way than -c command does.

  -v <path>         Look for duplicated files using current database.

                    Results will be placed into found.log.

  -i <database>     Look for new files comparing current database
                    and external database.

                    Results will be placed into found.log.

                    That command is intended to compare two different
                    databases. It won't support -k or -0 switches.

  -o                Optimizes current database.

                    This is a must when you use databases from other
                    weeder (such as VirWeeder Plus), but not really needed
                    if you use a database created with fweeder. Indeed you'll
                    get better performance if you optimize the database with
                    great amount of files.

                    You should use it before -a, -v or -i, and it won't
                    support -k or -0 switches.

                    If you notice fweeder spends too much time loading the
                    database should be a nice idea to optimize the database.

                    With this new database you will avoid fweeder's worst
                    case. Just think 1024 records needs about 10 tests in
                    worst case for one adding with an optimized database,
                    and up to 512 tests if the database is the worst
                    database possible.

                    You must use -x to optimize crc32 databases and in the
                    same way don't use -x with MD5 databases. Fweeder will
                    report such mistakes.

  available switch list:

  -d <database>     Use database as current database
  -k                Delete duplicated files
  -0                Delete zero size files
  -n                Use normal priority for fweeder thread
  -s                Detailed scan (for use with -c, -a and -v)
  -b                Beep at exit

  -x                Use CRC32 + file size instead of MD5

                    CRC32 is ok for enough people due it's fast, even
                    is less secure than MD5. I do not recommend its use
                    unless the machine you're using is very slow.
                    
                    Fweeder is intended to manage both MD5 and CRC32 +
                    file size databases in the right way, so if you don't
                    remember to put the -x switch while addind files to
                    a CRC32 database, fweeder will add it / remove it for
                    you as needed.
  
                    Notice databases in CRC32 form ARE NOT compatible with
                    VirWeeder databases. Fweeder is only compatible with
                    VirWeeder Plus (using MD5).

fweeder uses by default 'current' as database name, so you'll find following
files (if you're not using -d switch):

  current.db        database
  current.log       report (-c, -a)
  found.log         report (-v, -i)

Examples:

  fweeder -c c:\folder

  Creates a database for c:\folder using 'current' as db name.

  fweeder -a c:\folder -d mdb -k

  Adds files found into c:\folder using 'mdb' as db name, duplicated files
  will be deleted.

  fweeder -v c:\tmp -k

  Looks for duplicated files into c:\tmp using current database, duplicated
  files will be deleted.

  fweeder -c \collection -d my_collection -k -0

  Creates a database for \collection directory in current drive (the drive
  won't be added, so you can work with relative paths). The database will
  be my_collection.db and the report will be in my_collection.log.
  Duplicated files and zero size files will be deleted.

  fweeder -i external -d my_database

  Looks for new files into external.db database using my_database.db
  for the test. Results will be placed into found.log.

  fweeder -c d:\files -d files -k -0 -x -s

  Creates a database for d:\files folder using files as database (files.db)
  and the report will be into files.log. Duplicated files and zero size
  files will be deleted. CRC32 will be used instead MD5. The directory names
  will be shown in the screen due detailed scan switch.


IMPORTANT: Notice '.db' is the std ext for databases and fweeder will append
it to the name provided using -d switch. If you wanna use VIRWEEDP.CRC,
just rename it to VIRWEEDP.DB. Notice also that VirWeeder Plus should
support many hash functions, be sure the database was generated using MD5.


        4. History
        
2.0 - released 23 September 2002

Why version 2.0? Well, fweeder is pretty mature. I've added almost all the
features i think a file weeder must have. It's stable and ready for daily
development. With this version i've added CRC32 support to make happy the
people that like to use that hash. That's an important change in the tool,
so i wanna distinguish it from previous versions.

o -p switch removed and -n switch added. I'm not happy with this change, but
it's needed. You know there are other weeders out there, and if you check two
and fweeder is slower, you won't use it. The problem is most users don't
use -p switch while other weeders use highest priority by default. Now
fweeder uses it by default too, but i've added a switch (-n) to run the
weeder without steroids. With -n it can be, in some cases, about 30% slower.
However it's interesting if you wanna run the weeder in background in a
low resource machine (like mine).

o -s switch added. It provides a detailed scan: you can see directory names
while scanning.

o -b switch added. Fweeder beeps when the tool exits. Nice when you run
it in a minimized window.

o CRC32 + file size support added: -x switch. Now you can use CRC32 + size
instead MD5 as hash function. Even the database format is quite the same,
fweeder checks stored data to guess which hash was used. Many thanks to
Roadkill, VirusBuster, Perikles and VirusP for their help while testing
the CRC32 stuff.

o Little fixes in path processing.

o Bug fix in optimizer.

1.6 - released 29 August 2002

o Little (but important after all) bug fixed. Thanx to Yello and VirusBuster
for their reports.

1.5 - released 30 Juny 2002

o Some changes oriented to gain more speed, i586 opcodes used so keep safe
your 1.4 version for 386/486 computers. Indeed check that new version ;)

o New switch: -p. Now you can run fweeder at highest priority using this
switch. It can be applied to any command. The program will be more time
in the CPU, so be careful if you're running lots of apps and use this
switch.

1.4 - released 12 may 2002

o New -i command. Now you can compare two different databases looking for
  new files. This command is for traders: you can send a database for
  the files you want to exchange and the other collector can check it
  using his database.

o Little improvements loading databases.

1.3 - released 10 may 2002

o Fixed issue #004. Now works great under Windows XP. Win2k and WinNt not
  tested, but let's say it works (until someone reports error). Notice i
  cannot support systems i have not available for testing.

o Some little bug fixes for rare cases.

1.2 - released 4 may 2002

o Many optimizations.

o Fixed issue #003. Now -o command is available to optimize databases, so
  you can use VirWeeder Plus databases without problems.

1.1 - released 1 may 2002

o Documentation moved to an external file (this file!).

o Many little optimizations.

o Now elapsed time calculation is more accurate (i don't know why loading
  database was excluded, and the result was wrong most of times). Thanks to
  VirusBuster for the tips coding the time routines :)

o Added -0 switch that deletes files with zero size.

o Fixed issue #002. Zero size files now are managed better (the program will
  show it's ignoring a file, but won't appear the awful error message).

o Fixed issue #001. Spaces in path works great.

o Fixed issue #000. Now adding files *should* be faster working with a db
  created by fweeder. If you use a database optimized for VirWeeder Plus,
  just wait some time because that's the worst case again (it needs about
  one minute to rebuild a database with 60000 files). Issue #003 is still
  there :/ Now i know how to make a 'complete tree' to optimize addition,
  but the algorithm is pretty complex, so i'll add it to next release. By
  now is more important release a fix for severe issues (#000 mainly).

1.0 - released 28 apr 2002

o First release


        5. Known Issues
        
2.0 for next 2.1

There are not known issues.

*** *** *** *** *** *** *** *** *** *** *** ***

