snowflake data warehouse date range based joins 45x speedup

I have been using http://www.snowflake.net for a new data processing at work for a few months, and it’s just amazing, to be able to run large queries over large data-sets, and the ability to increase the cluster size, when doing development work to get faster turnarounds, that are not impacting the production cluster, brilliant.

One of the thing I have noticed is slower than I would like is joins based on tableA.time being inside a time-range of tableB.start and tableB.end when the time period being is in the months not days.

So the pattern mapping a value from TABLE_I onto all rows in the time span (not including the end)

CREATE OR REPLACE TEMPORARY TABLE WORKING_B AS
SELECT tp.u_id, tp.time, i.value
FROM TABLE_P tp
JOIN TABLE_I i ON tp.u_id = i.u_id
AND tp.time >= i.start_time AND tp.time < i.end_time;

For one set of data spanning 3 months the above takes 45 minutes on a small cluster for TABLE_P 65M/TABLE_I 10M rows.
Where-as for a similar set of 4 days, and ~45M rows this takes 30 seconds.

So I add some TO_DATE(time), TO_DATE(start_time) columns to the two tables, and then added AND tp.time_day = i.start_time_day and the first query went to ~60 seconds. But I was missing a few million rows as my time ranges span multiple days…

So I did many things that didn’t work (like trying to use a GENERATOR with dynamic input) and settled on a simple solution

CREATE OR REPLACE TABLE TEN_YEARS_OF_DAYS(date) AS
SELECT DATEADD(day, (rn-1),DateADD(months,4,DATEADD(years,-10,CURRENT_DATE))) FROM (
SELECT row_number() over(order by 1) as rn
FROM TABLE(GENERATOR(rowCount => 365*10)) v);

CREATE OR REPLACE FUNCTION get_dates_for_N_days ( start_date DATE, end_date DATE )
RETURNS TABLE (date DATE)
AS 'SELECT date FROM TEN_YEARS_OF_DAYS WHERE date BETWEEN start_date AND end_date';

so this creates a table with ten years of data (moved 4 months into the future) and a table function that selects the rows from it, so I can do a lateral join on that function

CREATE OR REPLACE TEMPORARY TABLE TABLE_I_B AS
SELECT e.*, t.date as range_day_part
FROM TABLE_I e, LATERAL get_dates_for_N_days(TO_DATE(e.start_time), TO_DATE(e.end_time)) t;

So the above code creates another temp table with a row per table B record with every Day is a duplicate row, now we have more rows in the seconds table, but we can do a date based match to speedup the query.

CREATE OR REPLACE TEMPORARY TABLE WORKING_B_B AS
SELECT tp.u_id, tp.time, i.value
FROM TABLE_P_B tp
JOIN TABLE_I_B i
ON tp.u_id = i.u_id AND tp.time_day = i.range_day_part
AND tp.time >= i.start_time AND tp.time < i.end_time;

This code runs in 60 seconds and gives the same results as the 45 minute code.

Things to note, putting LATERAL table joins on a selects with CTE’s presently breaks the SQL parser, in fact even nested selects and LATERAL don’t mix, thus the extra tables with _B etc. Also CTE’s make life so much easier, but as you start joining to them a lot, performance slips, I have found where I do a complex join is a good time to output to a temporary table, and the performance again is crazy…

Snowflake is just a beast. I have a man crush…

The Wrong Kind of Paranoia

James has a new post called http://prog21.dadgum.com/206.html in which he wonders if lots of the idioms like const, and static or sealed classes are them to allow developers to protect themselves from themselves. I would have commented on his blog, but he doesn’t like comments, which is a completely sane stance to have, heck the only comments I get here are spam.

So I had always thought those idioms where there to allow complier optimizations because there was extra information that was not tracked on older compilers, or to make the code cleaner.

const variables are to a avoid macro’s “true evil” and to keep the type information.
static allowed to keep the symbol table size down
sealed allowed real optimization, as functions will never to replaced.

And these were needed, so the new language could perform “better” at some key benchmark used to once again prove that poor “real world” problems in assembly, C are faster than in new language X. Thus all problems should use C/asm.

Goodbye Google Code, Hello GitHub

Well with the news Google is closing down Google Code, I’ve moved the two project I am responsible for to GitHub.

so https://code.google.com/p/coab/ is now at https://github.com/simeonpilgrim/coab

and https://code.google.com/p/nikon-firmware-tools/ is now at https://github.com/simeonpilgrim/nikon-firmware-tools

I’ve swapped over to the git tools, which I dislike compared to the hg tools, but I couldn’t get hg-git to play nice with github, so have moved on.

Tiny Planets Coding

The other day on Google+ I read a post from Ana Andres talking about coding Tiny Planets, and how it was running really slow in Matlab somewhere in the pixel drawing code.

That got me thinking about how I would code it in C#, and Ana had a good blog post showing one way of thinking about how the transformation was happening , one of her goals was to make an animated GIF so people could see the wrap bend happening.

Tiny Planet Code A
Tiny Planet A GIF

Now the this was plenty fast enough, I was getting 2-3 fps on my laptop using only one core, so from a speed perspective moved on.

My first plan was to interpolate between the points (all though that would really slow, and look bad at the outside), but when I showed my co-worker my method, he said I was silly and should just traverse the output space, and find the input pixel that maps, so I did that and got:

Tiny Planet Code C

I then had to put the bend code back in, as that was a special aspect needed for the animation and got:

Tiny Planet D GIF

I then started playing with sub-sampling to make the output less jaggie (F), and then I decided blend in YUV (or YCrCb) colour space, and finally settled on five point YUV average where you have a center of the pixel and the four 1/4 towards the pixel diagonal corners. Givening:

Tiny Planet H GIF

Sorry for GIF being links they are larger than they should be for in-lining..

Now improvement on the process would be to allow scaling and translation of the origin on the polar focus, this might be useful in an interactive UI to allow exploring the space.

Here’s my source code. A to H is the progression, it was fun to learn to make GIFs in C#.

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Drawing;
using System.Diagnostics;
using System.Windows.Media.Imaging;
using System.Windows;
using System.IO;

namespace TinyPlanet
{
	class Program
	{
		static void Main(string[] args)
		{
			Bitmap src = new Bitmap(@"C:\temp\tinyplanet-1.png");

			//GifA(src);

			//Bitmap dsta = Bend_A(src, i);
			//dsta.Save(@"C:\temp\tinyplanet-out_A.png");

			//Bitmap dstc = Bend_C(src, 100);
			//dstc.Save(@"C:\temp\tinyplanet-out_C.png");

			//GifD(src);

			//Bitmap dstd = Bend_D(src, 60);
			//dstd.Save(@"C:\temp\tinyplanet-out_D.png");

			//Bitmap dste = Bend_E(src, 60);
			//dste.Save(@"C:\temp\tinyplanet-out_E.png");

			//Bitmap dstf = Bend_F(src, 60);
			//dstf.Save(@"C:\temp\tinyplanet-out_F.png");

			//Bitmap dstg = Bend_G(src, 60);
			//dstg.Save(@"C:\temp\tinyplanet-out_G.png");

			//Bitmap dsth = Bend_H(src, 60);
			//dsth.Save(@"C:\temp\tinyplanet-out_H.png");

			GifH(src);
		}

		static void GifA(Bitmap src)
		{
			GifBitmapEncoder gEnc = new GifBitmapEncoder();

			for (int i = 0; i <= 100; i++)
			{
				Bitmap dst = Bend_A(src, i);
				Debug.WriteLine(string.Format("frame {0}", i));

				var frame = System.Windows.Interop.Imaging.CreateBitmapSourceFromHBitmap(
						dst.GetHbitmap(),
						IntPtr.Zero,
						Int32Rect.Empty,
						BitmapSizeOptions.FromEmptyOptions());
				gEnc.Frames.Add(BitmapFrame.Create(frame));
			}

			gEnc.Save(new FileStream(@"C:\temp\tinyplanet-out_A.gif", FileMode.Create));
		}

		static void GifD(Bitmap src)
		{
			GifBitmapEncoder gEnc = new GifBitmapEncoder();

			for (int i = 0; i <= 100; i++)
			{
				Bitmap dst = Bend_D(src, i);
				Debug.WriteLine(string.Format("frame {0}", i));

				var frame = System.Windows.Interop.Imaging.CreateBitmapSourceFromHBitmap(
						dst.GetHbitmap(),
						IntPtr.Zero,
						Int32Rect.Empty,
						BitmapSizeOptions.FromEmptyOptions());
				gEnc.Frames.Add(BitmapFrame.Create(frame));
			}

			gEnc.Save(new FileStream(@"C:\temp\tinyplanet-out_D.gif", FileMode.Create));
		}

		static void GifH(Bitmap src)
		{
			GifBitmapEncoder gEnc = new GifBitmapEncoder();

			for (int i = 0; i <= 100; i++)
			{
				Bitmap dst = Bend_H(src, i);
				Debug.WriteLine(string.Format("frame {0}", i));

				var frame = System.Windows.Interop.Imaging.CreateBitmapSourceFromHBitmap(
						dst.GetHbitmap(),
						IntPtr.Zero,
						Int32Rect.Empty,
						BitmapSizeOptions.FromEmptyOptions());
				gEnc.Frames.Add(BitmapFrame.Create(frame));
			}

			gEnc.Save(new FileStream(@"C:\temp\tinyplanet-out_H.gif", FileMode.Create));
		}

		static Bitmap Bend_A(Bitmap src, int bend_i)
		{
			int src_half_width = src.Width / 2;
			int dst_width = (src.Height * 2) + src.Width;
			int dst_height = src.Height * 2;
			int dst_origin_x = dst_width / 2;
			int dst_origin_y = dst_height / 2;
		
			double bend = bend_i / 100.0; // turn to percentage;

			int bend_x = (int)(src_half_width * (1.0 - bend));
			double bent_pixels = src_half_width - bend_x;
			double final_ang_d = 180.0 * bend;
			//Debug.WriteLine(string.Format("final_ang_d {0}", final_ang_d));
			double final_ang = final_ang_d * (Math.PI / 180.0);


			Bitmap dst = new Bitmap(dst_width, dst_height);
			Graphics g = Graphics.FromImage(dst);
			g.Clear(Color.White);
			g.DrawImage(dst, 0, 0, dst.Width, dst.Height);


			for (int x = 0; x < src_half_width; x++)
			{
				if (x >= bend_x)
				{
					// do polar
					int bx = x - bend_x;

					double rad = ((bx / bent_pixels) * final_ang) + (Math.PI / 2.0);
					//Debug.WriteLine(string.Format("bend % {0} ang {1}", bx / bent_pixels, rad * (180 / Math.PI)));
					double sin = Math.Sin(rad);
					double cos = Math.Cos(rad);


					for (int y = 0; y < src.Height; y++)
					{
						dst.SetPixel(dst_origin_x - bend_x + (int)(cos * y), dst_origin_y - (int)(sin * y), src.GetPixel(src_half_width - x, src.Height - y - 1));
						dst.SetPixel(dst_origin_x + bend_x - (int)(cos * y), dst_origin_y - (int)(sin * y), src.GetPixel(src_half_width + x, src.Height - y - 1));
					}
				}
				else
				{
					// do rectilinear
					for (int y = 0; y < src.Height; y++)
					{
						// left side
						dst.SetPixel(dst_origin_x - x, dst_origin_y - y, src.GetPixel(src_half_width - x, src.Height - y - 1));

						// right side
						dst.SetPixel(dst_origin_x + x, dst_origin_y - y, src.GetPixel(src_half_width + x, src.Height - y - 1));
					}
				}
			}

			return dst;
		}

		static Bitmap Bend_C(Bitmap src, int bend_i)
		{
			int src_half_width = src.Width / 2;
			int dst_width = (src.Height * 2) + src.Width;
			int dst_height = src.Height * 2;
			int dst_origin_x = dst_width / 2;
			int dst_origin_y = dst_height / 2;

			double bent_pixels = src_half_width;
			double final_ang = Math.PI;


			Bitmap dst = new Bitmap(dst_width, dst_height);
			for (int x = 0; x < dst_width; x++)
			{
				for (int y = 0; y < dst_height; y++)
				{
					// map from output to input
					int dx = x - dst_origin_x;
					int dy = y - dst_origin_y;

					double r = Math.Sqrt((dx * dx) + (dy * dy));
					double q = Math.Atan2(dy, dx);

					double pic_ang = q + (Math.PI / 2.0);
					double mod_ang = ((pic_ang + Math.PI) % (Math.PI * 2.0)) - Math.PI;

					int dev_x = (int)((mod_ang / final_ang) * bent_pixels);
					int dev_y = (int)r;

					//Debug.WriteLine(string.Format("x {0} y {1} dx {2} dy{3}, r {4}, q {5}, dev_x {6}, pic_ang {7}", x, y, dx, dy, r, q, dev_x, pic_ang));

					if (Math.Abs(dev_x) <= src_half_width &&
						dev_y < src.Height)
					{
						dst.SetPixel(x, y, src.GetPixel(dev_x + src_half_width, src.Height - dev_y - 1));
					}
				}
			}

			return dst;
		}

		static Bitmap Bend_D(Bitmap src, int bend_i)
		{
			int src_half_width = src.Width / 2;
			int dst_width = (src.Height * 2) + src.Width;
			int dst_height = src.Height * 2;
			int dst_origin_x = dst_width / 2;
			int dst_origin_y = dst_height / 2;

			double bend = bend_i / 100.0; // turn to percentage;


			int bend_x = (int)(src_half_width * (1.0 - bend));
			double bent_pixels = src_half_width - bend_x;
			double final_ang_d = 180.0 * bend;
			//Debug.WriteLine(string.Format("final_ang_d {0}", final_ang_d));
			double final_ang = final_ang_d * (Math.PI / 180.0);


			Bitmap dst = new Bitmap(dst_width, dst_height);
			Graphics g = Graphics.FromImage(dst);
			g.Clear(Color.White);
			g.DrawImage(dst, 0, 0, dst.Width, dst.Height);

			int bend_start_x = dst_origin_x - bend_x;
			int bend_end_x = dst_origin_x + bend_x;

			for (int x = 0; x < dst_width; x++)
			{
				int fix_x = (x < dst_origin_x ? bend_x : -bend_x);
				for (int y = 0; y < dst_height; y++)
				{
					if (x > bend_start_x &&
						x < bend_end_x)
					{
						// rectanliner 
						int ox = (x - bend_start_x) + (src_half_width - bend_x);

						if (y < src.Height)
						{
							dst.SetPixel(x, y, src.GetPixel(ox, y));
						}
					}
					else
					{
						// map from output to input
						int dx = x - dst_origin_x + fix_x;
						int dy = y - dst_origin_y;

						double r = Math.Sqrt((dx * dx) + (dy * dy));
						double q = Math.Atan2(dy, dx);

						double pic_ang = q + (Math.PI / 2.0);
						double mod_ang = ((pic_ang + Math.PI) % (Math.PI * 2.0)) - Math.PI;

						if (Math.Abs(mod_ang) <= final_ang)
						{
							int dev_x = (int)((mod_ang / final_ang) * bent_pixels) - fix_x + src_half_width;
							int dev_y = (int)r;

							//Debug.WriteLine(string.Format("x {0} y {1} dx {2} dy{3}, r {4}, q {5}, dev_x {6}, pic_ang {7}", x, y, dx, dy, r, q, dev_x, pic_ang));

							if (dev_x < src.Width && dev_x >= 0 && 
								dev_y < src.Height)
							{
								dst.SetPixel(x, y, src.GetPixel(dev_x, src.Height - dev_y - 1));
							}
						}
					}
				}
			}

			return dst;
		}

		static Bitmap Bend_E(Bitmap src, int bend_i)
		{
			int src_half_width = src.Width / 2;
			int dst_width = (src.Height * 2) + src.Width;
			int dst_height = src.Height * 2;
			int dst_origin_x = dst_width / 2;
			int dst_origin_y = dst_height / 2;

			double bend = bend_i / 100.0; // turn to percentage;


			int bend_x = (int)(src_half_width * (1.0 - bend));
			double bent_pixels = src_half_width - bend_x;
			double final_ang_d = 180.0 * bend;
			//Debug.WriteLine(string.Format("final_ang_d {0}", final_ang_d));
			double final_ang = final_ang_d * (Math.PI / 180.0);


			Bitmap dst = new Bitmap(dst_width, dst_height);
			Graphics g = Graphics.FromImage(dst);
			g.Clear(Color.White);
			g.DrawImage(dst, 0, 0, dst.Width, dst.Height);

			int bend_start_x = dst_origin_x - bend_x;
			int bend_end_x = dst_origin_x + bend_x;

			for (int x = 0; x < dst_width; x++)
			{
				int fix_x = (x < dst_origin_x ? bend_x : -bend_x);
				int ox = (x - bend_start_x) + (src_half_width - bend_x);

				for (int y = 0; y < dst_height; y++)
				{
					if (x > bend_start_x &&
						x < bend_end_x)
					{
						// rectanliner 
						if (y < src.Height)
						{
							dst.SetPixel(x, y, src.GetPixel(ox, y));
						}
					}
					else
					{
						// map from output to input
						int dx = x - dst_origin_x + fix_x;
						int dy = y - dst_origin_y;

						dst.SetPixel(x, y, Color.FromArgb(ARGBAtPoint(dx, dy, final_ang, ref src, bent_pixels, fix_x)));
					}
				}
			}

			return dst;
		}

		static Bitmap Bend_F(Bitmap src, int bend_i)
		{
			int src_half_width = src.Width / 2;
			int dst_width = (src.Height * 2) + src.Width;
			int dst_height = src.Height * 2;
			int dst_origin_x = dst_width / 2;
			int dst_origin_y = dst_height / 2;

			double bend = bend_i / 100.0; // turn to percentage;


			int bend_x = (int)(src_half_width * (1.0 - bend));
			double bent_pixels = src_half_width - bend_x;
			double final_ang_d = 180.0 * bend;
			//Debug.WriteLine(string.Format("final_ang_d {0}", final_ang_d));
			double final_ang = final_ang_d * (Math.PI / 180.0);


			Bitmap dst = new Bitmap(dst_width, dst_height);
			Graphics g = Graphics.FromImage(dst);
			g.Clear(Color.White);
			g.DrawImage(dst, 0, 0, dst.Width, dst.Height);

			int bend_start_x = dst_origin_x - bend_x;
			int bend_end_x = dst_origin_x + bend_x;

			for (int x = 0; x < dst_width; x++)
			{
				int fix_x = (x < dst_origin_x ? bend_x : -bend_x);
				int ox = (x - bend_start_x) + (src_half_width - bend_x);

				for (int y = 0; y < dst_height; y++)
				{
					if (x > bend_start_x &&
						x < bend_end_x)
					{
						// rectanliner 
						if (y < src.Height)
						{
							dst.SetPixel(x, y, src.GetPixel(ox, y));
						}
					}
					else
					{
						// map from output to input
						int dx = x - dst_origin_x + fix_x;
						int dy = y - dst_origin_y;

						int p0 = ARGBAtPoint(dx - 0.5, dy - 0.5, final_ang, ref src, bent_pixels, fix_x);
						int p1 = ARGBAtPoint(dx - 0.5, dy + 0.5, final_ang, ref src, bent_pixels, fix_x);
						int p2 = ARGBAtPoint(dx + 0.5, dy - 0.5, final_ang, ref src, bent_pixels, fix_x);
						int p3 = ARGBAtPoint(dx + 0.5, dy + 0.5, final_ang, ref src, bent_pixels, fix_x);

						int R = (((p0 >> 16) & 0xFF) + ((p0 >> 16) & 0xFF) + ((p0 >> 16) & 0xFF) + ((p0 >> 16) & 0xFF)) / 4;
						int G = (((p0 >> 8) & 0xFF) + ((p0 >> 8) & 0xFF) + ((p0 >> 8) & 0xFF) + ((p0 >> 8) & 0xFF)) / 4;
						int B = ((p0 & 0xFF) + (p0 & 0xFF) + (p0 & 0xFF) + (p0 & 0xFF)) / 4;

						dst.SetPixel(x, y, Color.FromArgb(R, G, B));
					}
				}
			}

			return dst;
		}

		static Bitmap Bend_G(Bitmap src, int bend_i)
		{
			int src_half_width = src.Width / 2;
			int dst_width = (src.Height * 2) + src.Width;
			int dst_height = src.Height * 2;
			int dst_origin_x = dst_width / 2;
			int dst_origin_y = dst_height / 2;

			double bend = bend_i / 100.0; // turn to percentage;


			int bend_x = (int)(src_half_width * (1.0 - bend));
			double bent_pixels = src_half_width - bend_x;
			double final_ang_d = 180.0 * bend;
			//Debug.WriteLine(string.Format("final_ang_d {0}", final_ang_d));
			double final_ang = final_ang_d * (Math.PI / 180.0);


			Bitmap dst = new Bitmap(dst_width, dst_height);
			Graphics g = Graphics.FromImage(dst);
			g.Clear(Color.White);
			g.DrawImage(dst, 0, 0, dst.Width, dst.Height);

			int bend_start_x = dst_origin_x - bend_x;
			int bend_end_x = dst_origin_x + bend_x;

			for (int x = 0; x < dst_width; x++)
			{
				int fix_x = (x < dst_origin_x ? bend_x : -bend_x);
				int ox = (x - bend_start_x) + (src_half_width - bend_x);

				for (int y = 0; y < dst_height; y++)
				{
					if (x > bend_start_x &&
						x < bend_end_x)
					{
						// rectanliner 
						if (y < src.Height)
						{
							dst.SetPixel(x, y, src.GetPixel(ox, y));
						}
					}
					else
					{
						// map from output to input
						int dx = x - dst_origin_x + fix_x;
						int dy = y - dst_origin_y;

						int p0 = ARGBAtPoint(dx - 0.5, dy - 0.5, final_ang, ref src, bent_pixels, fix_x);
						int p1 = ARGBAtPoint(dx - 0.5, dy + 0.5, final_ang, ref src, bent_pixels, fix_x);
						int p2 = ARGBAtPoint(dx + 0.5, dy - 0.5, final_ang, ref src, bent_pixels, fix_x);
						int p3 = ARGBAtPoint(dx + 0.5, dy + 0.5, final_ang, ref src, bent_pixels, fix_x);

						int yy, cr, cb;
						int yt = 0, crt = 0, cbt = 0;
						YCbCrFromRGB(p0, out yy, out cr, out cb);
						yt += yy; crt += cr; cbt += cb;
						YCbCrFromRGB(p1, out yy, out cr, out cb);
						yt += yy; crt += cr; cbt += cb;
						YCbCrFromRGB(p2, out yy, out cr, out cb);
						yt += yy; crt += cr; cbt += cb;
						YCbCrFromRGB(p3, out yy, out cr, out cb);
						yt += yy; crt += cr; cbt += cb;

						dst.SetPixel(x, y, Color.FromArgb(ARGBFromYCbCr(yt/4, cbt/4, crt/4)));
					}
				}
			}

			Debug.WriteLine(string.Format("red {0:x4}", Color.Red.ToArgb()));

			return dst;
		}

		static Bitmap Bend_H(Bitmap src, int bend_i)
		{
			int src_half_width = src.Width / 2;
			int dst_width = (src.Height * 2) + src.Width;
			int dst_height = src.Height * 2;
			int dst_origin_x = dst_width / 2;
			int dst_origin_y = dst_height / 2;

			double bend = bend_i / 100.0; // turn to percentage;


			int bend_x = (int)(src_half_width * (1.0 - bend));
			double bent_pixels = src_half_width - bend_x;
			double final_ang_d = 180.0 * bend;
			//Debug.WriteLine(string.Format("final_ang_d {0}", final_ang_d));
			double final_ang = final_ang_d * (Math.PI / 180.0);


			Bitmap dst = new Bitmap(dst_width, dst_height);
			Graphics g = Graphics.FromImage(dst);
			g.Clear(Color.White);
			g.DrawImage(dst, 0, 0, dst.Width, dst.Height);

			int bend_start_x = dst_origin_x - bend_x;
			int bend_end_x = dst_origin_x + bend_x;

			for (int x = 0; x < dst_width; x++)
			{
				int fix_x = (x < dst_origin_x ? bend_x : -bend_x);
				int ox = (x - bend_start_x) + (src_half_width - bend_x);

				for (int y = 0; y < dst_height; y++)
				{
					if (x > bend_start_x &&
						x < bend_end_x)
					{
						// rectanliner 
						if (y < src.Height)
						{
							dst.SetPixel(x, y, src.GetPixel(ox, y));
						}
					}
					else
					{
						// map from output to input
						int dx = x - dst_origin_x + fix_x;
						int dy = y - dst_origin_y;

						int p0 = ARGBAtPoint(dx - 0.25, dy - 0.25, final_ang, ref src, bent_pixels, fix_x);
						int p1 = ARGBAtPoint(dx - 0.25, dy + 0.25, final_ang, ref src, bent_pixels, fix_x);
						int p2 = ARGBAtPoint(dx + 0.25, dy - 0.25, final_ang, ref src, bent_pixels, fix_x);
						int p3 = ARGBAtPoint(dx + 0.25, dy + 0.25, final_ang, ref src, bent_pixels, fix_x);
						int p4 = ARGBAtPoint(dx, dy, final_ang, ref src, bent_pixels, fix_x);

						int yy, cr, cb;
						int yt = 0, crt = 0, cbt = 0;
						YCbCrFromRGB(p0, out yy, out cr, out cb);
						yt += yy; crt += cr; cbt += cb;
						YCbCrFromRGB(p1, out yy, out cr, out cb);
						yt += yy; crt += cr; cbt += cb;
						YCbCrFromRGB(p2, out yy, out cr, out cb);
						yt += yy; crt += cr; cbt += cb;
						YCbCrFromRGB(p3, out yy, out cr, out cb);
						yt += yy; crt += cr; cbt += cb;
						YCbCrFromRGB(p4, out yy, out cr, out cb);
						yt += yy; crt += cr; cbt += cb;

						dst.SetPixel(x, y, Color.FromArgb(ARGBFromYCbCr(yt / 5, cbt / 5, crt / 5)));
					}
				}
			}

			Debug.WriteLine(string.Format("red {0:x4}", Color.Red.ToArgb()));

			return dst;
		}

		static void YCbCrFromRGB(int RGB, out int Y, out int Cb, out int Cr)
		{
			int r = (RGB >> 16) & 0xFF;
			int g = (RGB >> 8) & 0xFF;
			int b = RGB & 0xFF;
			Y = (int)(0.299 * r + 0.587 * g + 0.114 * b);
			Cb = (int)(128 - 0.169 * r - 0.331 * g + 0.500 * b);
			Cr = (int)(128 + 0.500 * r - 0.419 * g - 0.081 * b);
		}

		static int ARGBFromYCbCr(int Y, int Cb, int Cr)
		{
			int r = Y + (int)((1.4f * (Cb - 128))+0.5);
			int g = Y + (int)((-0.343f * (Cr - 128) - 0.711f * (Cb - 128))+0.5);
			int b = Y + (int)((1.765f * (Cr - 128)) + 0.5);
			return (int)(0xFF000000 + (r << 16) + (g << 8) + (b));
		}


		static int ARGBAtPoint(double dx, double dy, double final_ang, ref Bitmap src, double bent_pixels, int fix_x)
		{
			double r = Math.Sqrt((dx * dx) + (dy * dy));
			double q = Math.Atan2(dy, dx);

			double pic_ang = q + (Math.PI / 2.0);
			double mod_ang = ((pic_ang + Math.PI) % (Math.PI * 2.0)) - Math.PI;

			if (Math.Abs(mod_ang) <= final_ang)
			{
				int dev_x = (int)((mod_ang / final_ang) * bent_pixels) - fix_x + (src.Width/2);
				int dev_y = (int)r;

				//Debug.WriteLine(string.Format("x {0} y {1} dx {2} dy{3}, r {4}, q {5}, dev_x {6}, pic_ang {7}", x, y, dx, dy, r, q, dev_x, pic_ang));

				if (dev_x < src.Width && dev_x >= 0 &&
					dev_y < src.Height)
				{
					return src.GetPixel(dev_x, src.Height - dev_y - 1).ToArgb();
				}
			}

			return Color.White.ToArgb();
		}
	}
		
}

Using Premake and GCC then avoid FloatFast

We are porting our MSVC/Win32 applications to Clang/GCC/Linux and have just spent the morning tracing why our unit tests fail.

void BadExampleCode()
{
    double a = NAN;
    ASSERT(!isnan(a));
}

Under MSVC and Clang all good, GCC asserts. We added printf’s and looked at the assembly and the code was hard coded to 0.

Some googling found 2006 posts stating GCC -ffast-math did odd things with isnan, and it’s still a presently reported issue

This came about because we are using Premake, and had the FloatFast flag set, because that’s how our MSVC projects were set, and we don’t want to change those builds, so for now we have tweaked the Premake code for this flag under GCC, as it doesn’t make sense that you would ever want isnan to be hard coded to zero, that’s really not fast maths at all.

We also put a #ifdef for ffast-math to break if this flag is present in the future.

And by we I mean Dave, and I just sat and talking it though with him.