This is not (yet) a complete patch, it is thought as a 
workspace for improvements to 
tools/freeze/moduulefinder, so that it can eventually go 
into the standard library.

Logged In: YES 
user_id=11105

First question for anyone listening: isn't the windows 
_try_registry stuff unneeded, because imp.find_module 
handles this itself? Even in python15, imp.find_module
("pythoncom") returns 'c:\\winnt\\system32\\pythoncom15.dll' 
on my system.

Logged In: YES 
user_id=11105

Assign to Mark, hoping for an answer.

Logged In: YES 
user_id=11105

Mark writes in private email: "Modules listed in the registry 
was a dumb idea.  This whole scheme can die.
AFAIK, no one in the world uses it (including win32all since 
the last build)."

Logged In: YES 
user_id=11105

I've removed in current CVS the support for modules listed in 
the registry.

Logged In: YES 
user_id=92689

Here's a patch. Summary: any_missing() returns less bogus
modules.

- I've rewritten scan_code() more or less from scratch,
factored bits and pieces out for readability.
- keep track of global assignments and failed imports per
module; use this to determine whether the Y in "from X
import Y" is a submodule or just a global name. This is not
100% doable: you can't tell which symbols are imported when
doing a star import of a non-Python module short of actually
importing it.
- added a new method to ModuleFinder: any_missing_maybe(),
which returns *two* lists, one with certain misses, one with
possible misses. The possible misses are *very* often false
alarms, so it's useful to keep this list separate.
any_misses() now simply returns the union of
any_missing_maybe().

I pretty sure this is all 100% backward compatible, so after
some more testing it could be checked in.

Logged In: YES 
user_id=92689

Oh, I just realized ModuleFinder.report() should still be
patched to use any_missing(), or even better,
any_missing_maybe(). Will do if this gets through.

Logged In: YES 
user_id=11105

Running the current CVS ModuleFinder on a simple script 
which contains 'import os' with Python 2.2 on Win2k finds 27 
modules, while with this patch it only finds 13.

(and unassign Mark)

Logged In: YES 
user_id=92689

Whoops, that was a useless patch :-( I managed to forgot to
copy the last three lines of the original scan_code() which
takes care of delayed imports... Fixed in the new patch.

I now also patched Modulefinder.report() to actually use the
new any_missing_maybe() feature.

(Note for the lib doco to be written: we should document
that people should use any_missing[_maybe]() instead of
looking directly at mf.badmodules. mf.badmodules is the raw
list, any_missing_maybe() filters out the false positives.)

I've ran before/after tests on all of Lib/*.py and
Lib/test_*.py and the only differences I found are those
that this patch address, ie. less false positives when
reporting missing modules.

Logged In: YES 
user_id=11105

Now that it works really fine, I would like to make progress.
This new patch 'thomas1.patch' is against the patched 
modulefinder, it is easier to read this way.

First it passes all changes to self.badmodules through the 
_add_badmodule() method.

Then, and this part is not really polished, but you may get the 
idea, it introduces a new ignored instance variable which is 
thought to suppress most of the remaining missing modules 
warnings.

Logged In: YES 
user_id=92689

What's the difference between this "ignore" variable and the
"exclude" list? And why do you need to ignore modules based
on importing module? In my OSX appbuilder stuff I simply
have a list of modules that will be removed from the
'missing' list when reporting:

MAYMISS_MODULES = ['mac', 'os2', 'nt', 'ntpath', 'dos',
'dospath',
	'win32api', 'ce', '_winreg', 'nturl2path', 'sitecustomize',
	'org.python.core', 'riscos', 'riscosenviron', 'riscospath'
]

(I think I could just add this to the exclude list and get
the same effect.)

Let's please keep it simple!

Logged In: YES 
user_id=92689

Thomas, your _add_badmodule patch actually broke my code...
It took me a while to figure out why, and while at it, I
discovered a blatant stupidity in my own patch: somehow I
completely overlooked the fact that mf.badmodules already
references the modules that did the failed import, so my
module.badimports thing can go. It does the exact same
thing. Almost... and your patch made that clear: the lines
that you patched actually make the badmodules reference lie
(and with your patch it also makes mod.badimports lie ;-):
if module X imports Y.Z, and Z isn't found, it lists Y as an
importer of Y.Z, which isn't true. My any_maybe_missing()
code needs to know whether the __init__.py tried to failed
import _itself_, because if it did (and failed) it's certain
that the submodule is missing. But if it _didn't_, the
"missing submodule" should be listed as "maybe missing", as
it's most likely just a global in the __init__.py. I'll try
to revise my patch tomorrow, it should become simpler.

In the meantime I think it might indeed be useful to
distinguish between "may be missed" and "must be excluded".
Is that what your "ignore" patch tried to do?

Logged In: YES 
user_id=11105

Just, I trust your analysis while I don't understand it ;-).
Thinking more about my "ignore" concept shows that it is 
unneeded. Warnings can be better removed after 
modulefinder has been run by examining the data it collects.

To answer your question: "excludes" does not mean 'may be 
missed', it means 'must be excluded' IMO, and no, this was 
not the purpose of my patch.

Logged In: YES 
user_id=92689

Here's a revised patch. The previous version actually was
too lenient: it could list submodules as "maybe missing"
when they were missing for sure. Fixed by tracking
unresolvable starimports.

(Removed the module.badimports attribute of my previous
patch, it now uses mf.badimports appropriately.)

Meta question: are we going to commit fixes to
modulefinder.py in the freeze directory and do the move when
the doco is done, or do we not commit anything until we're
ready for the move?

Logged In: YES 
user_id=33168

What's the status of this patch now that modulefinder.py has
been moved into the library?

Logged In: YES 
user_id=11105

I think it can be closed (although the docs are still missing).