Why Killing Processes may be Hurting You and What to Do About it

First the Background:
A while back I wrote an insane program to automate Omnipage 14 (an OCR program) mostly via GUI automation. The urge to do this was caused by Omnipage not handling more than 300 files on its own before dying, and the lack of a better solution in the market place.
So Omnipage was very tempermental and quit often for all kinds of unexplained reasons. So my solution was to feed Omnipage 3 files at once and when it finished OCR’ing the 3 files kill all of its processes. Then give it 3 new files and relaunch it and so on.

Once I had completed the programming I set it off on it’s own to work on a network drive full of ~30,000 files needing to be OCR’d.

1 month goes by …

I come back to check on it and here is what I see on the screen!
System Tray Chaos
Apparently killing the processes didn’t allow some Omnipage process to clear its system tray icon. So as the headline promised here is at least one way that killing a process is harmful (and humorous). I’m sure there are plenty of more serious reasons though. (Yes, moving the mouse over these icons caused them to dissapear, so I spent a challenging 30 minutes randomly moving my mouse around the system tray and exterminating these things … )

A Better Option Than Killing
After that experience, here is the method I switched to. After Omnipage finishes its 3 files, I kindly request that it close itself using Python’s Win32 extensions. Here is the code (mostly copied from their win32 examples):

import sys
import win32process, win32api, win32pdhutil, win32con, win32security
import win32event, msvcrt, win32gui

def get_all_windows():
    """Returns dict with window desc and hwnd"""
    def _MyCallback( hwnd, extra ):
        """Helper function"""
        hwnds, classes = extra
        hwnds.append(hwnd)
        classn = win32gui.GetClassName(hwnd)
        if not classn in classes:
            classes[classn] = []
        classes[classn].append( hwnd )
    windows = []
    classes = {}
    win32gui.EnumWindows(_MyCallback, (windows, classes))
    return classes

def request_windows_to_close(list_window_descs_to_close):
    """Send me a list of stuff to close such as:stuff_to_close=['OP15_OmniPage_Agent_Class']
    run get_all_windows() to see what's available"""
    classes=get_all_windows()
    for windesc_to_close in list_window_descs_to_close:
        if windesc_to_close in classes:
            for hwnd in classes[windesc_to_close]:
                win32gui.PostMessage( hwnd, win32con.WM_CLOSE, 0, 0 )
                print 'requested closure with', windesc_to_close
        else:
            print 'Did not find a window desc. as',windesc_to_close

Life is good now I think :-)

Points of discussion:

  1. Did I miss an OCR product that I can just give it a directory with up to 60K files and have it OCR them into PDF’s?
  2. Is there a programmatic way to ask Windows to refresh the system tray (Don’t say moving the mouse programmatically over the system, that’s cheating.)
  3. I considered using the Omnipage SDK instead of GUI automation but it looked too expensive and hard to learn. Has anyone had experience with that method?

[tags]Python, win32, python win32, Python for Windows, Omnipage, GUI Automation, Automation, WTF, Windows 2000, System tray, icon, WM_CLOSE, win32gui, OCR, Autoit[/tags]

6 Responses to “Why Killing Processes may be Hurting You and What to Do About it”

  1. Paul Ford says:

    After writing similar code to kill OmniPage on a regular basis, and going out of my mind because something always went a little wrong (OmniPage 15 is just as bad–Nuance has failed completely), I’m having good luck with ReadIRIS Pro Corporate. The OCR requires more training–I initially rejected the program because the OCR produced more errors than OmniPage, but gave it another look after spending several months desperately trying to make OmniPage behave. ReadIris does not catch as many edge cases as OmniPage, but it’s accurate enough for my needs, and it has a good deskew algorithm, which, since my output is PDF image-on-text, makes for much easier reading.

    The FineReader Batch tool works more or less as you’d expect it to–so far I’ve found that it can convert several thousand images in a go without sucking down more than 100 megs of memory. I have 250K scans as medium-sized sets of 600 DPI jpegs in 1800 directories, so I: scale down to 300DPI, use Jpeg2PDF (http://koan.studentenweb.org/software/jpeg2pdf.html) (with one line hacked to keep the page size [inches] accurate at 300DPI) to compile one PDF for each directory, then batch the PDFs. The result is one nicely OCRed and straightened 100+ page PDF out for each PDF in. It just chugs along; after dealing with OmniPage I keep expecting it to explode but it works as advertised, and I get about 316 pages an hour on a 2.8GHz Xeon.

    Adobe Acrobat and Abbyy FineReader both use some old-school bitonal deskew that leaves color images looking like they’ve been stomped on, although FineReader does fine with sequential batches of up to exactly 10K pages each and is perfect for bitonal scans; it’s also cheaper than ReadIris Corporate.

  2. Paul,

    Thanks for the very informative comment. I’m surprised that there’s another “Omnipage Killer” out there but I guess I shouldn’t be. I’ll definately look into your suggestions if I get that frusterated with Omnipage, life is good after I started to ask it nicely to close, perhaps that was the final issue …

  3. boz says:

    Abbyy Finereader has a quality OCR engine; PDF image support is somewhat iffy… Your mileage will vary based on the quality of your scans , the consistency thereof, and the much time / effort you spend in templating them such that the ocr engine can make better guesses.

  4. Boz, I believe I tried the Finereader and it almost worked out but still was crashing and bugging out.

    Ah well, I finally got Omnipage 15 running indefinately now through an eleborate system of GUI automation, Python, and various DOS process killing utilities ;-) It took a year of identifying every possible way it could fail and expecting it, but we’re good now. I’ll have to make a post about that sometime.

  5. mike hibbard says:

    Greg,

    As someone who isn’t looking forward to a similar journey with omnipage (50,000 multi page image only pdfs to ocr with Omnipage15) I would be very appreciative of your working method.

    Regards

    Mike

  6. Mike,

    I feel your pain. It may be worth looking at the Omnipage SDK before you embark on a journey of programming something similar to mine. Since making a program like mine might take you a year it’s worth spending even two months learning the SDK! Keep me updated.