Re: [Openexr-devel] Speeding up Python

[tyler]

Ah, that's copying from one imath array to another, and that's not what I'm trying to accomplish.

I'm trying to get the data out of the imath array objects into another form (in this case, numpy)

I know the imathnumpy module exists. However, the code only handles 2 of the array types IIRC.

It also requires the compiled module to be linked with numpy, and numpy isn't always available like in Windows Maya, or if I wanted to use the built-in python array.array, or my own custom module.

~T.Fox

On Mon, Apr 1, 2019 at 1:00 PM Nick Rasmussen <address@hidden> wrote:

The native imath python arrays are pretty fast, I think your version is triggering an element by element python copy of data into the numpy array.  Here's the timings with the native (fast) array copy and the python only iteration on the array types.  Here's the script:

import imath

import time

count = 1000000

ai = imath.V3dArray(imath.V3d(0.25, 0.3, 0.75), count)

ao = imath.V3dArray(count)

start = time.time()

ao[:] = ai

end = time.time()

print 'Copying %d elements: %0.06f seconds' % (count, end-start)

start = time.time()

for i in range(count):

    ao[i] = ai[i]

end = time.time()

print 'Python iteration %d elements: %0.06f seconds' % (count, end-start)

And the output:

> python2 foo.py

Copying 1000000 elements: 0.003245 seconds

Python iteration 1000000 elements: 0.994242 seconds

Does that match what you're seeing?  That's also without enabling the multithreading that's built into the array operations.

-nick

On Mon, Apr 1, 2019 at 12:46 PM Tyler Fox <address@hidden> wrote:

Getting data in and out of imath in python is *SLOW*. And yes, Python is a slow language, but that doesn't mean we can't kick it up a notch.

Could I get some feedback on my proposals to speed things up?

https://github.com/openexr/openexr/pull/373

First is a PR with a *very* simple change that just exposes the memory address of the imath object.

With a few lines of python code, you can read the c-data and re-interpret it in whatever way you want.

https://github.com/openexr/openexr/issues/367 

This is an issue where I described a more complex (and maybe more pythonic?) way of exposing the underlying data.

Either of these proposals could remove the need for imathnumpy... Or imathnumpy could be replaced with a pure-python module that handles the more error prone memory mapping.

Thoughts?

~T.Fox

_______________________________________________
Openexr-devel mailing list
address@hidden
https://lists.nongnu.org/mailman/listinfo/openexr-devel