Primary example:

Python 2.3.5 (#1, Mar 20 2005, 20:38:20) 
[GCC 3.3 20030304 (Apple Computer, Inc. build 1809)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> 0 ** 0
1

Per http://mathworld.wolfram.com/Indeterminate.html, 0 ** 0 is an indeterminate form. 0 ** 0 should probably return NaN and NOT 1.

Other examples include:

Python 2.3.5 (#1, Mar 20 2005, 20:38:20) 
[GCC 3.3 20030304 (Apple Computer, Inc. build 1809)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> inf = float("infinity")
>>> inf ** 0
1.0
>>> 1 ** inf
1.0

For a few others, Python provides an arguably correct answer of NaN. Examples:

Python 2.3.5 (#1, Mar 20 2005, 20:38:20) 
[GCC 3.3 20030304 (Apple Computer, Inc. build 1809)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> inf = float("infinity")
>>> inf * 0
nan
>>> inf / inf
nan
>>> inf - inf
nan

And, of course, for the most obvious indeterminate form (0/0) Python does precisely the right thing:

Python 2.3.5 (#1, Mar 20 2005, 20:38:20) 
[GCC 3.3 20030304 (Apple Computer, Inc. build 1809)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> 0/0
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
ZeroDivisionError: integer division or modulo by zero

It could be argued that the correct thing to do here is to throw an exception - mathematically speaking these forms are about as meaningful as division by zero which Python handles by throwing an exception. (unlike Java and  IEEE 754, which do arguably evil things here by returning +/- infinity for the quantity k/0 for all k != 0)

Unfortunately, some people doing numerical work may have gotten used to the current Python behavior so the change isn't without risk. And some of the current values are common "conventions" used to simplify certain common issues. (e.g. 0 ** 0 == 1) Moreover, I don't know if this is dependent on platform (having only tried it on a couple of the boxes I have handy.) 

However, from a mathematical purist's standpoint, Python is doing the wrong thing on these counts.

Python's behavior with respect to floating point arithmetic is left to the platform's C floating point libraries.  For example, on my Windows machine running Python 2.3.5, float("infinity") raises a ValueError.  I would also point out that 0/0 is integer division in Python 2.3.5 .

For other examples of platform-specific behavior:
Python 2.3.5 (#62, Feb  8 2005, 16:23:02) [MSC v.1200 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> inf*0
-1.#IND
>>> inf**0
1.0
>>> 1**inf
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
ValueError: (33, 'Domain error')
>>> inf*0
-1.#IND
>>> inf/inf
-1.#IND
>>> inf-inf
-1.#IND
>>>

So yeah.  If you don't like how Python does math, complain to your vendor (Apple) or compile a version of Python with a C floating point library that works the way you want it to.

Definition: n**m == 1 multiplied by n m times.
This definition is consistent with n**m * n**k == n**(m+k).
By this definition, the 3 examples you call wrong are correct,
though the latter two involving inf are possibly platform dependent.
Closing as invalid