This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: super(...).__new__( ... ) behaves "unexpected"
Type: Stage:
Components: Interpreter Core Versions: Python 2.4
process
Status: closed Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: brenck, jimjjewett, ncoghlan
Priority: normal Keywords:

Created on 2005-03-16 17:07 by brenck, last changed 2022-04-11 14:56 by admin. This issue is now closed.

Files
File name Uploaded Description Edit
run.py brenck, 2005-03-16 17:07 python code AND trace output
Messages (12)
msg24651 - (view) Author: Dirk Brenckmann (brenck) Date: 2005-03-16 17:07
Hello there,




python code and trace output enclosed.




What I did:


1. Metaclass inheritence:


   type <-- MA <-- MB <-- MC




2. Created Class A using __metaclass__= MC




3. Create Class B(A) using __metaclass__= MB




...although this might seem strange, it should work...




When taking a look at the trace, you will notice one line 
that goes like:


'-------> why does super( MA, metacls ).__new__ call MC.
__new__ in next line ????????????????????'




if you run the code, you will find it three times. That's ok. 
In my trace I just replaced two occurences of that line by 
">>" to enable focussing on the Problem...




What I would expect the code to do is the following:


1. Create a Class A which is of type MC


2. Create a Class B(A) which is of type MB




What the interpreter does is different:


1.   Create a Class A which is type MC


2.1 Nearly create a Class B which is of type MB.


2.2 In type.__new__( ... ) change it's mind.


2.3 Due to the superclass A<MC> of B, create some 
class A which is of type MC as well. Although  B 
contains a __metaclass__ = MB statement.




Well - that's what I experienced is "the way windows 
works", so I ran the code on Solaris again but the 
behaviour remains reproduceable...




I would consider it a bug therefor. If it's not a bug, I would 
expect an Exception which tells me where I did wrong...




Thanx for your time and efforts


   


msg24652 - (view) Author: Dirk Brenckmann (brenck) Date: 2005-03-16 17:11
Logged In: YES 
user_id=360037

Sorry - 2.3 must be corrected:




2.3 Due to the superclass A<MC> of B, create some 


class B which is of type MC as well. Although B 


contains a __metaclass__ = MB statement.
msg24653 - (view) Author: Dirk Brenckmann (brenck) Date: 2005-03-18 10:38
Logged In: YES 
user_id=360037

Ok - I think I found the reason for the behaviour I explained 
above ...

typeobject.c:
  function type_new(...)
   [...]
   /* Determine the proper [...]
  <line 1611, 1612, 1613>
    if (PyType_IsSubtype(tmptype, winner)) {
	winner = tmptype;
	continue;
    }

These three lines mean, it would never be possible to create a 
subclass using a (metatype which is a subclass of the 
metatype) of it's superclass.
In such cases, (even) a metaclass (explictly set by a 
programmer) is ignored and replaced by a metaclass which 
python decides...

... do you really want this to be that way????

This means, one always ends up in a (meta-)class hierarchy 
that is fixed. Programmers dynamically can't decide anymore, 
if there should be a level in their class hierarchy which should 
*NOT* use the 'highest subclassed metaclass' in their class 
hierarchy.

I would consider this a problem, because the use of 
__metaclasses__ will be implicitly restricted
msg24654 - (view) Author: Dirk Brenckmann (brenck) Date: 2005-03-18 14:13
Logged In: YES 
user_id=360037

Ok - I think I found the reason for the behaviour I explained 
above ...

typeobject.c:
  function type_new(...)
   [...]
   /* Determine the proper [...]
  <line 1611, 1612, 1613>
    if (PyType_IsSubtype(tmptype, winner)) {
	winner = tmptype;
	continue;
    }

These three lines mean, it would never be possible to create a 
subclass using a (metatype which is a subclass of the 
metatype) of it's superclass.
In such cases, (even) a metaclass (explictly set by a 
programmer) is ignored and replaced by a metaclass which 
python decides...

... do you really want this to be that way????

This means, one always ends up in a (meta-)class hierarchy 
that is fixed. Programmers dynamically can't decide anymore, 
if there should be a level in their class hierarchy which should 
*NOT* use the 'highest subclassed metaclass' in their class 
hierarchy.

I would consider this a problem, because the use of 
__metaclasses__ will be implicitly restricted
msg24655 - (view) Author: Dirk Brenckmann (brenck) Date: 2005-03-18 14:41
Logged In: YES 
user_id=360037

Ok - I think I found the reason for the behaviour I explained 
above ...

typeobject.c:
  function type_new(...)
   [...]
   /* Determine the proper [...]
  <line 1611, 1612, 1613>
    if (PyType_IsSubtype(tmptype, winner)) {
	winner = tmptype;
	continue;
    }

These three lines mean, it would never be possible to create a 
subclass using a (metatype which is a subclass of the 
metatype) of it's superclass.
In such cases, (even) a metaclass (explictly set by a 
programmer) is ignored and replaced by a metaclass which 
python decides...

... do you really want this to be that way????

This means, one always ends up in a (meta-)class hierarchy 
that is fixed. Programmers dynamically can't decide anymore, 
if there should be a level in their class hierarchy which should 
*NOT* use the 'highest subclassed metaclass' in their class 
hierarchy.

I would consider this a problem, because the use of 
__metaclasses__ will be implicitly restricted
msg24656 - (view) Author: Dirk Brenckmann (brenck) Date: 2005-03-18 15:48
Logged In: YES 
user_id=360037

Ok - I think I found the reason for the behaviour I explained 
above ...

typeobject.c:
  function type_new(...)
   [...]
   /* Determine the proper [...]
  <line 1611, 1612, 1613>
    if (PyType_IsSubtype(tmptype, winner)) {
	winner = tmptype;
	continue;
    }

These three lines mean, it would never be possible to create a 
subclass using a (metatype which is a subclass of the 
metatype) of it's superclass.
In such cases, (even) a metaclass (explictly set by a 
programmer) is ignored and replaced by a metaclass which 
python decides...

... do you really want this to be that way????

This means, one always ends up in a (meta-)class hierarchy 
that is fixed. Programmers dynamically can't decide anymore, 
if there should be a level in their class hierarchy which should 
*NOT* use the 'highest subclassed metaclass' in their class 
hierarchy.

I would consider this a problem, because the use of 
__metaclasses__ will be implicitly restricted
msg24657 - (view) Author: Jim Jewett (jimjjewett) Date: 2005-03-18 19:14
Logged In: YES 
user_id=764593

Yes, it is intentional.  

class Derived(Base): ...

should mean that you can use an instance of Derived 
anywhere you need an instance of Base.

There are ways to break this, but it isn't a good idea.

Letting Derived ignore part of the metaclass (and creation 
instructions) of Base would make it much easier to create an 
invalid Derived by accident -- and the error could show up as 
memory corruption, instead of something meaningful.
msg24658 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2005-03-19 00:14
Logged In: YES 
user_id=1038590

Minimal case that should throw an exception on C3, but
doesn't (instead, it silently replaces the explicitly
requested metaclass M1 with its subclass M2):

class M1(type):
  pass

class M2(M1):
  pass

class C1(object):
  __metaclass__ = M1

class C2(C1):
  __metaclass__ = M2

class C3(C2):
  __metaclass__ = M1
msg24659 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2005-03-19 00:17
Logged In: YES 
user_id=1038590

To address the documentation side, the following text from
Guido's descrintro essay could be added to the official
documentation:

"""For new-style metaclasses, there is a constraint that the
chosen metaclass is equal to, or a subclass of, each of the
metaclasses of the bases. Consider a class C with two base
classes, B1 and B2. Let's say M = C.__class__, M1 =
B1.__class__, M2 = B2.__class__. Then we require
issubclass(M, M1) and issubclass(M, M2). (This is because a
method of B1 should be able to call a meta-method defined in
M1 on self.__class__, even when self is an instance of a
subclass of B1.)""" 
msg24660 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2005-03-19 01:30
Logged In: YES 
user_id=1038590

Additional text from the descrintro essay, indicating that
it is deliberate that C3 does not trigger an exception:

"""However, if one of the base metaclasses satisfies the
constraint (including the explicitly given __metaclass__, if
any), the first base metaclass found satisfying the
constraint will be used as the metaclass."""

So, unless Guido chooses to change the desired behaviour,
these two snippets pretty much cover what needs to be added
to the docs.
msg24661 - (view) Author: Dirk Brenckmann (brenck) Date: 2005-03-19 10:32
Logged In: YES 
user_id=360037

Okay, that's the point.
Thanx for your time and work - bug becomes deleted.
msg24662 - (view) Author: Jim Jewett (jimjjewett) Date: 2005-03-20 01:25
Logged In: YES 
user_id=764593

Note that the documentation is known to be very weak 
regarding both metaclasses and the differences between 
classic and new-style classes.

If you (or Nick) wanted to submit a documentation patch, I 
doubt I'm the only one who would appreciate it.
History
Date User Action Args
2022-04-11 14:56:10adminsetgithub: 41706
2005-03-16 17:07:50brenckcreate