Demo of the upcoming release.

For those who asked to see my latest project example. Here it is, a work in progress that will be available soon.

Note: this is GIF animation, the original graphics is real-time 60 FPS.

Stay tuned.

Cheers.

 

Screen-Space Input to 3D Object Orientation

Input position (touch or mouse) can be converted back to 3D space by applying unprojection transformation to a 2D device coordinate. This is still problematic, because 3D frustum is squashed into xy-plane and is losing z-component.

Thus, reconstructing full 3D coordinate is not possible without additional step - detecting object intersection etc.

One way of doing it is to have distance to a target object serve as a billboard plane, then:

• using camera FOV, direction and orientation, calculate distances travelled by a pointer as projected onto a plane,
• convert those to a movement around a bounding sphere,
• construct quaternion orientations for origin and destination positions,
• apply constraints as needed (i.e. to make alignment with axis),
• use slerp or other interpolation methods if smooth animation is needed.

Some example code, in Swift:

    // Initial 2D device coordinate.
    func setStart(point: CGPoint) {
        let startCoordWorld = unprojectTouch(point: point)
        start = startCoordWorld
        next = startCoordWorld
        startOrientation = targetNode.simdOrientation.normalized
    } // FUNC SET START
    

    // Follow up 2D device coordinate.
    func setNext(point: CGPoint) {
        let nextCoordWorld = unprojectTouch(point: point)
        next = nextCoordWorld
        planarToSpherical()
        addOrientation = simd_quatf(angle: angleRad, axis: axis).normalized
    } // FUNC SET NEXT
    
    
    // Makes the node change orientation (corresponding to start-next pair).
    func jumpToNext() {
        targetNode.simdOrientation = (addOrientation * startOrientation)
    } // FUNC ORIENT TO NEXT
    

    // Convert viewport point to a world coordinate on an imaginary plane facing camera.
    func unprojectTouch(point: CGPoint) -> simd_float3 {
        let ndcPoint = viewToNDC(point: point)
        let xyPlanePoint = ndcPoint * fovWorldSize * 0.5
        
        // This point is on an imaginary plane that is perpendicular to a view.
        let worldPlaneTouchPoint = cameraPos
            + cameraUp * xyPlanePoint.y
            + cameraRight * xyPlanePoint.x
            + cameraFront * distanceCamToPlane
        
        return worldPlaneTouchPoint
    } // FUNC UNPROJECT TOUCH
    

    // View coordinates (screen pixels) to NDC (normalized device coordinates).
    // Only x and y can be recovered.
    func viewToNDC(point: CGPoint) -> simd_float2 {
        // Centered in the middle of view.
        let x = Float(point.x) - viewportSize.x * 0.5
        let y = viewportSize.y * 0.5 - Float(point.y)
        let ndc = simd_float2(x * 2.0 / projectionSize,
                              y * 2.0 / projectionSize)
        return ndc
    } // FUNC VIEW TO NDC


    // Convert plane movement in front of a camera to angular rotation.
    // Around "bounding" sphere of a target node.
    func planarToSpherical() {
        // Plane distance to the sphere center is targetRadius.
        let distance = simd_distance(start, next)
        angleRad = distance / (targetRadius)
        axis = simd_normalize(simd_cross(start, next))
    } // FUNC PLANAR TO SPHERICAL


    // Intermediate rotation (from current to next).
    // Input is normalized in range [0, 1]
    func interpolateOrientation(t: Float) {

        // slerp has a problem - always takes the shortest arc.
        var interOr = simd_bezier(Self.identityOrientation, Self.identityOrientation, addOrientation, addOrientation, t)
        interOr = (interOr * startOrientation)
        targetNode.simdOrientation = interOr
    } // FUNC INTERPOLATE ORIENTATION

Not everything is here, but the general thinking is like that.

To further improve on this - here is an idea - make the orientation snap to predetermined values, like in the following animation :):

Have fun.

Proposal: Autonomous UAV Navigation System

A proposal for a project to develop an advanced autonomous UAV navigation system.
Full description and development is on the project page:

http://alxyly.blogspot.com/p/uav.html

Proposal: Air Flow Simulation for Pandemic Control

A proposal for the CFD project to help with pandemic research.
Full description and development is on the project page:

http://alxyly.blogspot.com/p/air.html

ML with Python (numpy, pandas), my tools + code snippets (Part 1)

Contents:

1. Jupyter notebooks

2. Path to packages
3. PYZO editor for .py
4. Basics of numpy arrays
5. Numpy.random
6. Array operations
7. Boolean indexing
8. Fancy indexing
9. Transposition

1. Jupyter notebooks

Jupyter is part of Anaconda pack of packages, but I prefer simple commands:

$ python3 -m pip install jupyter

Jupyter notebooks run a web server that can be queried either directly from web console, or via terminal. To find what localhost:port is used by a running server:

$ jupyter notebook list

To stop a server:

$ jupyter notebook stop 8888 

To open a notebook file (with .ipynb extension), in terminal point to a notebook directory and run:

$ jupyter notebook

This starts a server, then opens a notebook's home directory in a browser. Click a notebook to run it. Keep terminal shell open and running a server as an attached process. To stop a server use Ctrl-C, or just close a terminal.

Shift-Return to run selected cell and advance to the next.

2. Paths to Python components and packages

>>> import sys  

>>> print('\n'.join(sys.path))

3. PYZO editor for .py

PYZO works for me. It is lightweight, has debugging options, and is configurable.
Supports cells, blocks of code that can be executed with a simple shortcut (Cmd-Return). Define cells by delimiting with #$$ optional commentary lines.

Some shortcuts (original and my own custom):

---------- EDITOR: 

Opt-Tab    - select previous file 

F1         - focus to shell panel 

F2         - focus to file editor 

Cmd-/      - comment selection  

Cmd-Opt-/  - uncomment  

---------- RUN:  

Cmd-R      - run file as a script (in console restarts interpreter)  

Cmd-Return - run cell with cursor (cells are delimited by #%%)   

Opt-Return - run selection  

---------- DEBUGGER:  

Cmd-B      - toggle breakpoint  

F6         - step over  

F7         - step in  

F8         - step out  

Ctrl-Cmd-Y - continue  

Ctrl-Cmd-. - stop debugging 

4. Basics of numpy arrays

Many ways to init arrays. There are two main options - shape and dtype.

>>> x = np.ndarray(shape=(2, 2), dtype=np.int8, order='C') 

>>> print(x) 

[[1 0] [1 1]]

Internal buffer is linear. Shape can be changed easily without reallocation if total element count remains the same:

>>> x.shape = (1,4) 

>>> print(x) 

[[1 0 1 1]]

Other array constructors:

>>> print(np.array((1, 2, 3))) 

[1 2 3] 

>>> print(np.zeros((2, 3))) 

[[0. 0. 0.] 

 [0. 0. 0.]] 

>>> print(np.empty((2,))) 

[7.74860419e-304 7.74860419e-304]

Array construction using list comprehensions. Note, unspecified dimension size of -1 infers the required element count in that dimension:

>>> x = np.array([(x, y) for x in [1,2,3] for y in [3,1,4] if x != y]) 

>>> print(x) 

[[1 3] 

 [1 4] 

 [2 3] 

 [2 1] 

 [2 4] 

 [3 1] 

 [3 4]] 

>>> x.shape = (2, -1) 

>>> print(x) 

[[1 3 1 4 2 3 2] 

 [1 2 4 3 1 3 4]] 

>>> x.shape = (-1) 

>>> print(x) 

[1 3 1 4 2 3 2 1 2 4 3 1 3 4]

Numpy arrays in list comprehension expressions:

>>> y = np.array([e for e in x if not e % 2]) 

>>> print(y) 

[4 2 2 2 4 4]

Range into 2D array:

>>> x = np.arange(15).reshape(5, -1).T 

>>> print(x) 

[[ 0 3 6 9 12] 

 [ 1 4 7 10 13] 

 [ 2 5 8 11 14]]

Numpy array operations perform like native C memory access ops (or parallelized vectors in GPU). Therefore much faster than Python's list comprehensions:

#%% Timing init 

import time 

x1 = np.arange(1000000) 

x2 = np.arange(1000000) 

#%% Timing native 

t0 = time.time() 

for _ in range(10): 

    x1 **= 2 

t1 = time.time() 

print("x1 time = ", t1 - t0) 

#%% Timing list comprehension 

t0 = time.time() 

for _ in range(10): 

    x2 = np.array([x ** 2 for x in x1]) 

t1 = time.time() 

print("x2 time = ", t1 - t0) 

#%%

---------------------------------------- 

x1 time = 0.015141725540161133 

x2 time = 4.744795083999634

5. Numpy.random

To generate standard numpy arrays filled with random numbers:

np.random.rand(d0,..dn)     - with each value uniformly in range [0, 1). 

np.random.randn(d0,..dn)    - with each value in Gaussian distribution, where mean = 0, variance = 1 (sigma squared). 

sigma * np.random.randn(d0,..dn) + mean    - a full normal distribution.

There are lots of other useful functions, i.e. np.random.choice(...).

6. Array operations

Slice of an array returns a data structure that defines subrange and points at the original array. Note that the second index (i.e. in [3:10]) points beyond the last element:

x1 = np.arange(12) 

print(x1) 

x1_slice = x1[3:10] 

print('x1_slice      = ', x1_slice) 

print('x1_slice[0:3] = ', x1_slice[0:3]) 

x1_slice[0:3] = 100 

print(x1) 

---------------------------------------- 

[ 0 1 2 3 4 5 6 7 8 9 10 11] 

x1_slice      = [3 4 5 6 7 8 9] 

x1_slice[0:3] = [3 4 5] 

[ 0 1 2 100 100 100 6 7 8 9 10 11]

Multi-dim array slices:

x1 = np.arange(12).reshape(4,-1) 

print(x1) 

x1_slice = x1[2:4] 

print(x1_slice) 

print(x1_slice[0][1:3]) 

---------------------------------------- 

[[  0  1  2] 

 [  3  4  5] 

 [  6  7  8] 

 [  9 10 11]]

[[ 6 7 8] 

 [ 9 10 11]] 

[7 8]

Slicing across multiple dimensions:

print(x1[:2, 1:3]) 

x1[:, 1:2] = 100 

print(x1) 

---------------------------------------- 

[[1 2] 

 [4 5]] 

[[  0 100  2]  

 [  3 100  5] 

 [  6 100  8] 

 [  9 100 11]]

7. Boolean indexing

Taken and modified from the pydata-book.

A boolean operation with array will return an array of boolean element-wise results. Boolean array when used as index will pick array elements where boolean component is True.

Boolean array size must be equal to data array size at the index:

names = np.array(['Bob', 'Joe', 'Will', 'Bob', 'Joe']) 

data = np.random.randint(1, 10, (5, 4)) 

print(names) 

print(data) 

mask = (names == 'Bob') | (names == 'Will') 

print(mask) 

print(data[mask]) 

data[mask, 1:3] = 100. 

print(data) 

data[data > 5] = 0 

print(data) 

----------------------------------------  

['Bob' 'Joe' 'Will' 'Bob' 'Joe'] 

[[1 4 3 6] 

 [6 1 3 3] 

 [7 1 6 1] 

 [3 3 9 4] 

 [4 7 8 9]] 

[ True False True True False] 

[[1 4 3 6] 

 [7 1 6 1] 

 [3 3 9 4]] 

[[  1 100 100   6] 

 [  6   1   3   3] 

 [  7 100 100   1] 

 [  3 100 100   4] 

 [  4   7   8   9]] 

[[1 0 0 0] 

 [0 1 3 3] 

 [0 0 0 1] 

 [3 0 0 4] 

 [4 0 0 0]]

8. Fancy indexing

Passing list of numbers into an indexer will treat it as an index picker. Allows assembling a new array from a combination of elements of another array:

x1 = np.arange(20).reshape((5, 4)) 

print(x1) 

x2 = x1[[1, 4, 2, 2], [0, 3, 1, 2]] 

print(x2) 

x1[[1, 4, 2, 2], [0, 3, 1, 2]] = 100 

print(x1) 

----------------------------------------   

[[ 0  1  2  3] 

 [ 4  5  6  7] 

 [ 8  9 10 11] 

 [12 13 14 15] 

 [16 17 18 19]] 

[ 4 19 9 10] 

[[  0   1   2   3] 

 [100   5   6   7] 

 [  8 100 100  11] 

 [ 12  13  14  15] 

 [ 16  17  18 100]]

9. Transposition

Transposed matrix is pointing at the same data (no copying takes place):

x1 = np.arange(15).reshape((3, 5)) 

print(x1) 

x2 = x1.T 

print(x2) 

x2[2] = 100 

print(x1) 

----------------------------------------    

[[ 0  1  2  3  4] 

 [ 5  6  7  8  9] 

 [10 11 12 13 14]] 

[[ 0 5 10] 

 [ 1 6 11] 

 [ 2 7 12] 

 [ 3 8 13] 

 [ 4 9 14]] 

[[  0  1 100  3  4] 

 [  5  6 100  8  9] 

 [ 10 11 100 13 14]]

References:

1. https://github.com/wesm/pydata-book
2. https://leemendelowitz.github.io/blog/how-does-python-find-packages.html

3. https://docs.python.org/3/installing/index.html

4. http://nbviewer.jupyter.org/github/pydata/pydata-book/blob/2nd-edition/ch02.ipynb#
5. https://docs.python.org/3/tutorial/datastructures.html#tuples-and-sequences

Microsoft's Imagine Cup Challenge - Team Heuristic Clinic

A month ago I prepared a quick team and a project for consideration at Microsoft's Imagine Cup. It was an intriguing experience to document details, to present in a video, and to polish a previously developed prototype - a wireless motion/gesture capturing app and a set of sensors.
I understood there was a slim chance to actually win the Cup, so I instead focused on setting up this project for possible future applications elsewhere.

The team is called Heuristic Clinic, the project is Applied Neural Diagnosis. Brief description can be found here: Heuristic Clinic, and the resulting certificate can be viewed here.
They were kind to offer $500 worth of Azure cloud, but I had to decline because the biggest expense is funding client-side development. Appreciated.

2018 Big Idea Challenge

Top 50 - 2018 Big Idea Challenge

This Certificate is Presented To:

alexey l

Team Heuristic Clinic

Congratulations and thank you for your hard work and dedication as a competitor in Imagine Cup 2018. You are now part of an elite international community of students who have shown remarkable creativity and innovation to push technology forward.

Pablo Veramendi
Imagine Cup Competition Manager
Microsoft Corporation

Martian sandstorm

Now a fully interactive volumetric sand storm, probably somewhere on Mars.
This marks an intermediate milestone following months of research into volume rendering. It uses finite differences solver for a fluid dynamics equation, some additional research into graphics pipeline on a Mac and iOS.
Note the shadows, they depend on volume density, and also note the unbounded nature of volumes. Other methods would produce visible banding artifacts.

Unexpected volumetric clouds

I was playing with code that I am testing to produce fluid dynamic effects, and I came across a weird cloud generating setup.
The advection part is failing at some point here and causes gas density to freeze in some regions of the map. It remains static and appears as a cloud, and I think this can be further improved to produce realistic looking real cloud. What's missing is a better phase function, and shading based on anisotropic scattering of different wavelenths.

Fun with self-shadowing volumetrics

As a preparation to a full-fledged interactive volume rendering, here is the first functional volumetric composition. It uses a few techniques to achieve a self-shadowing appearance, as you would expect from a real-life volume of transparent matter.

Volumetric hints.

A few volume rendering hints.

1. Cannot discard writing into render-targets selectively in an MRT frame-buffer. This arises, for example, when two targets are used to store two custom denormalized depth buffers. A frame-buffer has a depth attachment as well, but it is used in a regular sense of a depth buffer.

2. Volume rendering can use analytic volume buffer or a set of depth textures. To limit volume to a custom 3D shape requires preliminary step to render that shape into two depth attachments, one with inverted normals and inverted depth test:

// Enable depth test, inverted, front to back.
glDepthMask(GL_TRUE);
glEnable(GL_DEPTH_TEST);
glDepthFunc(GL_GREATER);

// Render only back faces.
glEnable(GL_CULL_FACE);
glCullFace(GL_FRONT);

// Clear the buffers.
// Note that the most distant shape will overwrite all others.
glClearDepth(0.0);
glClearColor(0.0f, 0.0f, 0.0f, 0.0f);
glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT);

//... Render the shape of the volume, i.e. a cloud.

// Enable depth test, back to front.
glDepthMask(GL_TRUE);
glEnable(GL_DEPTH_TEST);
glDepthFunc(GL_LESS);

// Render only front faces.
glEnable(GL_CULL_FACE);
glCullFace(GL_BACK);

// Clear the buffers.
glClearDepth(1.0);
glClearColor(0.0f, 0.0f, 0.0f, 0.0f);
glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT);

//... Render the same shape of the volume.

3. In a fragment shader calculate the distance to a rasterized point. Do it in fragment instead of in a vertex, hoping for speedup due to hardware interpolation. If two vertices spread out evenly in a view space, their depths will be equal and much bigger than the distance to a front clipping plane. This results in artifacts when moving towards a volume and crossing it.

4. A depth buffer attachment can now be shared between with other frame-buffers. It works for simple convex shapes as volumes that can be rendered from inside and outside. The real work is to be done elsewhere to modulate this shape with other detail, i.e. noise.