In this picture, you can assume the relationship between the

>>8420143
Wow, you answered your own question.

Great thread OP. I'm sad it's over.

if the numbers in one column do not repeat you find a polynomial of degree n where n is the size of the columns, that maps the first one to the other.

if you're a pleb tier programmer like me looking for a quick way to map x to y try this place:
https://mycurvefit.com/

>>8420154

Thanks, that site looks really useful. I will try to test it tomorrow maybe.

>>8420149
>you find a polynomial of degree n where n is the size of the columns
I think that would likely cause too much over fitting.

>>8420143
Linear regression is more powerful than that.
If you take the logarithm of both the set of y coordinates and the set of x coordinates, and take the linear regression of your new set of (log(x),log(y)) coordinates, you can look for power relations in your data set. This works because, if your formula is y = x^3, then your points are (log(x),log(y))=(log(x),log(x^3))=(log(x),3*log(x)), which is clearly a linear relation between log(x) and log(y) (and the slope of this linear regression is the degree of the polynominal).

>>8420231
But what if you have something like y=e^x

>>8420237
If you have an exponential pattern or a logarithmic pattern, then you will need to take the logarithm of the y coordinates, but not the x coordinates, or take the logarithm of the x coordinates, but not the y coordinates, respectively, then take the linear regression of your new data set.

>>8420143
>How would you find the relationship between two columns with several million numbers and no obvious sequence or pattern?

Naively:
1) Partition the data into (at least) a training set and an evaluation set.
2) Hypothesize first order models of Y's dependence on X
3) Train your models on the training data
4) Assign prior likelihood to each model in your set based on relative complexity
5) Update respective likelihood based on model's ability to match evaluation set
6) Discard models with posterior liklihood below some threshold
7) If desired, hypothesize next order corrections to remaining models and repeat 3-7

>>8420143
y = 2x

>>8420143
If you have two culumns, and you know that one column is a function of the other, than you can just fit it to a model, like a polynomial series or a fourier series, as long as the functions in your basis are orthogonal, you should be fine.

>>8420383
What if what you have is a wave or a tangent?