Issue472568
This issue tracker has been migrated to GitHub,
and is currently read-only.
For more information,
see the GitHub FAQs in the Python's Developer Guide.
Created on 2001-10-18 21:31 by anonymous, last changed 2022-04-10 16:04 by admin. This issue is now closed.
Messages (9) | |||
---|---|---|---|
msg7033 - (view) | Author: Nobody/Anonymous (nobody) | Date: 2001-10-18 21:31 | |
Memory buffer areas created by PyBuffer_New are missaligned on 32 bit machines, for doubles (64 bit). This typically generates a bus error crash on most RISC machines when a library function tries to write a double in a memory buffer allocated by PyBuffer_New(). When looking at the bufferobject.c code, this seems to come from the fact that the memory buffer points at 'malloc() + sizeof struct PyBufferObject'; [line: b->b_ptr = (void *)(b + 1); ] To the extent that struct PyBufferObject is not a multiple of sizeof (double), the largest value type; thus the misalignment, and the crashes on most RISC processors. A quick and temporary solution would consist in adding a dummy double field as the last field of the PyBufferObject structure: struct PyBufferObject { ... double dummy; } ; and setting b->b_ptr = (void*)&b->dummy; /*was (void*(b+1)*/ In doing so, b->b_ptr will always by aligned on sizeof (double), on either 32 and 64 bit architectures. Since I'm on the buffer type problem: It would be nice (and probably easy to do) to augment the buffer protocol interface with the ability to specify the basic value type stored in the buffer. A possible list of values (enum ...) would be: - undefined (backward compatibility) - char, unsigned char, short, ....int, ... long, long long, float, double, and void* (memory address). This would enable to check at runtime the type of values stored in a buffer (and prevent missalignement buserrors, as well as catching garbage situations when improper array types are passed by means of the buffer interface [e.g.: int/float/double/short...). Frederic Giacometti Frederic Giacometti |
|||
msg7034 - (view) | Author: Frederic Giacometti (giacometti) | Date: 2001-10-18 21:35 | |
Logged In: YES user_id=93657 I wasn't looged in when I submitted the item. Don't think I'm becoming anonymous :)) |
|||
msg7035 - (view) | Author: Jeremy Hylton (jhylton) | Date: 2001-10-22 20:19 | |
Logged In: YES user_id=31392 Note to whomever take this bug: PyBuffer_New() is not called anywhere in the Python source tree; nor are there any tests for buffer objects that I'm aware of. A few simple test cases would have caught this bug already. (And for the case of the builtin buffer() call, it might be good if it used PyBuffer_New().) |
|||
msg7036 - (view) | Author: Frederic Giacometti (giacometti) | Date: 2001-11-01 00:31 | |
Logged In: YES user_id=93657 A portable solution (im[provement over what I proposed) would consitst in declaring 'dummy' with a union type, 'unionizing' all C-ANSI value types (and including 'long long' optionally by mean of an #ifdef). { .... union { int Int; long Long; double Double; void* Pvoid ...} dummy; } All (void*)obj->dummy can be replaced with obj->dummy.Pvoid FG |
|||
msg7037 - (view) | Author: Tim Peters (tim.peters) * | Date: 2001-11-01 01:31 | |
Logged In: YES user_id=31435 The only portable way to fix this (assuming it's broken -- I don't see any alignment promises in the docs, and since we never use it we can't mine the source code for clues either) is to have PyBuffer_New do a second separate malloc (size) and set b_ptr to that. The C std guarantees memory returned by malloc is suitably aligned for all purposes; it doesn't promise that any standard type captures the strictest alignment requirement (indeed, at KSR we returned 128-byte aligned memory from malloc, to cater to our "subpage" type extension). |
|||
msg7038 - (view) | Author: Frederic Giacometti (giacometti) | Date: 2001-11-01 17:21 | |
Logged In: YES user_id=93657 The solution Tim proposes (2d malloc) will be very fine with what we do. Here are some more details on what we're doing (and this is a standard operation): - We want to create an array of double that we pass to a C function, and then return this array to Python as buffer object (the buffer is passed latter on as arg to other functions using the buffer interface); and do it so that Python takes ownership of the buffer memory management. - We don't want to require Numerical to operate the package; just for memory allocation. - We should actually be using the Python array module interface. Unfortunately: * the Python array object C definitions are not exported in a .h file * the Array python interface does not provide the ability to create a new array of an arbitrary size (and certainly initialized to 0). One has to provide a list or a string to create an array of a given size. IThis is not workable if we work we large arrays (e.g.: an array of 1.000.000 doubles is only 8 MB RAM ...). Another solution, then, would consist in extending the array Python interface, so as to enable the creation of arrays of arbitrary sizes (prefereably initialized to 0 or to something alse with a calloc or a memset). The extension of the array.array() function could be the better solution, taking into account our needs as well as Tim's concerns. FG |
|||
msg7039 - (view) | Author: Tim Peters (tim.peters) * | Date: 2002-06-17 15:30 | |
Logged In: YES user_id=31435 Unassigned -- I don't expect to work on this. |
|||
msg7040 - (view) | Author: Armin Rigo (arigo) * | Date: 2003-09-22 18:57 | |
Logged In: YES user_id=4771 No more activity around here. I suggest to deprecate this bug report. Here is a note for the docs saying the memory PyBuffer_New() gets you isn't specifically aligned. (cannot attach it, sorry) http://arigo.tunes.org/api_concrete.diff |
|||
msg7041 - (view) | Author: Neil Schemenauer (nascheme) * | Date: 2004-06-08 03:00 | |
Logged In: YES user_id=35752 Added note to documentation as suggested by Armin. |
History | |||
---|---|---|---|
Date | User | Action | Args |
2022-04-10 16:04:32 | admin | set | github: 35351 |
2001-10-18 21:31:53 | anonymous | create |