Quantitative tufts

Optical tuft tracking isn't exactly novel, as it already exists in research applications. It's likely been used on cars too, perhaps even beyond the wind tunnel during on-track sessions, but there's not much documentation for that. I figured I'd have a go at implementing some sort of tuft-tracking program anyway though, as trying to get useable tuft results from a single photo or video frame is both tedious, and inaccurate. A full video is more representative, but getting quantitative results from simply watching a video is impossible.

I started by looking into current methods of tuft tracking as seen in lab settings, and found good examples from Ma and Chen (2022), and Omata and Shirayama (2017). Unfortunately, neither of the methods used are suitable for imperfect test scenarios (especially regarding lighting and contrast). The first paper uses luminescent tufts in a dark environment to significantly increase natural contrast, with tuft detection based on bright pixels within the overall image. The second paper relies on intense external lighting to increase contrast, then employs an edge-detection method, which even in the lab conditions, was shown to be imperfect where the background was not homogenous. 

My first idea was motion identification, so I set up an environment in Matlab and gave the default motion ID process a test on what I considered to be a video in realistic/non-ideal conditions (low lighting, shadows, low resolution). For basically copying an existing template and feeding it the raw poor-quality footage the results were impressive. There was promising tuft identification, but the fundamental issue was that tufts that weren't moving much didn't get identified well. Obvious, right? I considered combining this method with one of the more typical methods (edge detection, template matching, etc), but there was no clear way of reliably linking the data from multiple methods like this. 

The image below shows the output, with red areas attempting to locate areas of motion, and green lines joining the furthest distance within each red area, thus creating a "tuft". 

Failing to find any suitbable methods out of the box, I decided to try making my own algorithm. It was based on the method used by Ma and Chen, but more robust to poor-quality recording, poor environmental conditions (contrast, shadows, varying background brightness), and able to accommodate backwards-facing tufts as a result of reversed local flow direction.

The first major difference was the requirement for the user to input tuft origin coordinates by manually clicking on each tuft in a UI. This adds an extra step, but the rewards are a significant freedom of how the algorithm then goes about detecting the rest of the tuft, which allows much more resilience to poor lighting. The hardest part of the tuft tracking is the initial identification of individual tufts against a complex background of lines and shadows and varying brightness, which this method avoids. Critically, the required accuracy of the user input click is low, both for successful tuft identification, and the accuracy of the resulting tuft angle estimate. This means clicking rapidly through tufts is possible; I was able to select two tufts per second during testing with adequate accuracy, meaning even large grids of hundreds of tufts can be selected in 1-2 minutes. This is a one-off process per tuft setup, as the inputs can be saved and applied to multiple videos of the same set of tufts. 

The blue circles indicate the initial search area and thus the start of each tuft must be within this region. 

After confirming the tuft selection and before running the algorithm, the user must also manually select a rectangular region that represents a stationary reference. This allows the program to determine global image shift due to camera vibration (critical for on-track vehicle use), and adjust the pixel coordinates for the tufts accordingly. In the example below, I've selected a region towards the top of the image that contains no tufts. A alternative option would be to place a small contrasting sticker somewhere on the surface, away from any tufts. 

The algorithm then loops through each tuft in each frame using the steps below. 

1. Create a vector of length = Search Radius (as per user input, and shown by the blue circles), anchored at the user-selected origin point.

2. Sweep the vector through a range of 45 degrees at 15 degree intervals either side of the initial tuft direction (also a user input value). This initial tuft direction value reduces the accuracy requirement of the initial tuft selection clicks, and relies on clear tape being used, guaranteeing that even if the flow is reversed, the first section of the tuft will always be in the expected direction. 

3. For each 15 degree interval, sample each pixel that the vector passes over, and record the average brightness.

4. Determine the direction with lowest or highest overall brightness, depending on the tuft and background colours

5. Move the active path point to the end of this vector

6. After the first step, reduce Search Radius automatically to two thirds of the input value, and allow the search vector to sweep through 60 degrees either side of the final vector angle from the previous step. Initially, this logic worked, but resulted in significant overshoots. Paths sometimes got lost, and there was increased error in the final angle estimate. The solution was to dynamically adjust the path step length depending on the best angle identified (smaller step if path curves more). This remained independent of the search radius. 

7. Adjust the current pixel coordinates to account for global frame shift due to camera vibration

8. After travelling the minimum distance (another user input), the average brightness of all pixels with a radius of Search Radius * 5 from the current path point is calculated, and used to dynamically adjust a threshold for identification of the end of a tuft. When the darkest 5 pixels in the vector search window are not at least 20% darker (or lighter if using white tufts) than the average of the larger surrounding area, then the path stops and the current point is designated as the end point of the tuft. 

9. The final tuft angle for each frame is taken as the angle of the vector between the tuft endpoint, and the point 40% of the way along the path. By taking the vector across the latter 60% of the tuft, effects of tuft stiffness are greatly reduced (and the section of tuft "trapped" under the clear tape is avoided). 

10. A tuft "curvature" metric is calculated, as the angle difference between two vectors extending from the 70% position (one back to the 40% mark, and one to the tuft endpoint/100% mark). Even in extreme cases such as reversed airflow, the trailing 60% of each tuft should be close to straight regardless of its direction, so this curvature metric is quickly able to identify tufts with questionably high curvature (suggesting the search algorithm may have become lost and wandered off the tuft), and the frame/s they occur on, for quick verification/debugging of the results. 

After tuning the tracking algorithm, it was time to turn the results into useable outputs. The first output is a vector field showing average tuft anlges over the duration of the input video. Interpolation between each tuft provides increased resolution. The risk of interpolation is that areas of complex flow may be overly-simplified and thus inaccurate, but this is ok as long as the user is aware of this limitation. Such awareness is increased through various other outputs included below.

For quantitative comparison of angles, the individual tuft vectors can be printed/exported, along with their coordinates. To improve usability of the outputs (both numerica and visual) it is possible to apply a distortion shift to the result to eliminate the effects of camera perspective and fish-eye. The use of a consistently-spaced grid of tufts will help provide suitable anchor points for such a transformation. 

The tufts I used in my tests were not ideal (high stiffness, not straight), and neither was the rather rough 3D printed surface that I had taped them to, so I wasn't able to judge quantitative accuracy very well. The tuft output and CFD L.I.C. are superimposed below on the top surface of a test part. 

Generally the results are pretty good, and discrepancies were entirely down to the experimental setup, not the tuft tracking. I didn't plan on trying to find the perfect wool or string for these initial tests; for now I just want the algorithm to work. Eventually I'll give the program to the FSAE team and they can test it for real. 

The next test used the same part/surfaces, but with an upstream obstruction moved from left to right, then back again. This was to test the capability of the algorithm to track more chaotic tufts.

The path tracking fell short of the ends of the tuft in a few places, but typically only where there was significant blur. These tests utilised only a 60 Hz frame rate, which was increased to 240 Hz for following tests to reduce motion blur. 

Next was a separation test, which utilised the suction side of the same test part. Separation can be identified visually with reasonable confidence using the average vector field. To increase reliability and automate the separation identification, a separate formula is used that combines spatial rate of change of average angle between neighbouring upstream tufts with a check for flow that at least occasionally (>5% of frames) has a negative stream-wise velocity component. If these two checks both return positive (or for the front-most tufts, only the negative velocity criteria is assessed), then the affected tufts are flagged as areas of separated flow. 

The flow along the edge shows strong vortex formation and so the separation is limited to near the front of the part before the vortex fully forms. 

The separation detection and average vector angles are superimposed in the second image below. The part was hand-held to further test the algorithm's ability to deal with vibration.

At this point I'd achieved the original goal of making a reliable tuft-tracking algorithm, and some results visualisations. I wanted to experiment further though and see how much information I could extract from these tufts. 

I started looking at whether I could get a rough idea of relative transience/turbulence in different parts of the flow. In total, I tested 7 different independent formulas for "turbulence":

• Total sum of angular travel of each tuft over all frames

• Total sum of angular change in curvature  of each tuft over all frames

• Angle of a window encompassing the median 50% of all instantaneous angles for each tuft

• RMS of the fluctuations in tuft tip velocity

• Count of oscillations of each tuft across its mean angle

• Average tuft oscillation frequency about its mean angle

• Sum of the power of all frequencies experienced by each tuft in the range of 1 to 10 Hz

The results were tested on a finite cylinder wake scenario (11cm diameter, 5cm height, Re = 50000 approx.). The low cylinder height was a mistake in the end as it meant the downstream flow was vortex-dominated instead of being a chaotic wake. Unfortunately I didn't have a taller cylinder to use at the time. 

CFD prediction for turbulent kinetic energy:

"Turbulence" formulas 1 and 2 (total motion):

"Turbulence" formulas 3 and 4:

"Turbulence" formulas 5, 6, and 7 (frequency-based):

There are two dominant patterns among the seven results, with one being more of a typical "wake" shape such as that normally given by total pressure contours, and the other shape (formulas 5 and 6), appears to be close to the TKE estimate, but inverted. This could easily be a coincidence at this point though, so much more testing would be needed and plenty of other formulas could be applied, but this shows the potential for the sort of information that could be extracted from tufts beyond just angles.

Later, I had a look for different thread and wool types that might improve the reliability of the results: cotton, polyester, polyethylene and silk, all with different strand weights and counts. The most promising was an elastic thread, which didn't hold bends or curves and was very flexible.

Elastic thread has good flexibility (left) and does not maintain bends or curvature (right), while remaining lightweight, low-profile, and easily visible. 

I ran another test, this time with a tall rectangular prism positioned upstream, using the elastic thread. In the end, pure cotton actually worked better. While the elastic thread was softer and more compliant, it suffered from three undesirable characteristics:

• It was slightly grippy and occasionally got stuck at lower velocities, even on smooth surfaces 

• The elastic nature meant there was a significant spring/inertia effect in areas of oscillating flow, and even in non-oscillating flow where the angle was more than about 20 degrees

• The cotton coating shifted around easily over the elastic core, resulting in inconsistent stiffness along the length of the tufts, and between different tufts. The bare elastic core had far too much friction to be usable without the cotton coating. 

As a result, angle estimations were less accurate, and turbulence estimations were too noisy to be interpretable. 

The main drawbacks of the pure cotton that led to my search for an alternative were the bends and kinks that it held. I expect pre-treating the thread by hanging it up with weights and steaming it will yield much straighter tufts and thus better results. 

At least by running the new elastic test, I was able to confirm the code still worked fine for light tufts on a dark background, even with very low contrast. The 47 tufts tracked below over 300 frames took about 10 seconds to compute on a single core. Parallel-processed "parfor" loops could be substituted both per-tuft and per-frame to significantly speed this up.