>spend a shitload of time getting up2date on machine learning,

>>60156720
come on, how hard is it to get a machine to play an audio sample of someone saying "do you want fries with that"?

>>60156720
I'm in the exact opposite boat.

I'm a genetic programming specialist, I've just been told to take two weeks to develop a proposal to do NLP and they think that Deep Learning is the best way to do it unless I come up with a good reason to do something else.

I've never actually written a neural net, let alone have any idea how to implement NLP with it or develop one into some kind of Deep Learning.

>>60156876
You laugh now, but some day they will recognize the brilliance of my waifu classifier

>>60159172
>NLP via RNNs

you're fucked, basically

>>60160786
>you're fucked, basically
I'm feeling a bit that way.

In DL you can mix techniques, provided they're vaguely amenable to layering so that's something. Not really sure how the fuck to do it though.

>>60160812
Does the company you work for have a shitload of usable text data available? Otherwise that'd be a pretty good argument for you to suggest something simpler

>>60160834
>Does the company you work for have a shitload of usable text data available
Yeah, we're not processing arbitrary text, it's technical documents, more or less and I think the collection already exists.

I guess what the project is supposed to do is read these paragraphs of text and extract key facts from them, then store those facts in a structured form (i.e. basically as rows in a table)

>>spend a shitload of time getting up2date on machine learning, deep learning especially
What did you study? Have you read deep learning book?

>>60160912
Is the data labeled?
I hate how "Use neural nets bro just add layers lol" has a huge survivor bias, as >>60160834 it's almost always easier and better to use a simpler model.

>>60161112
>Is the data labeled?
You mean the text? No, though I guess I will end up manually labeling some as a training set.

>>60161196
>No, though I guess I will end up manually labeling some as a training set.

That will take a fuck ton of time. This is the survivor bias that I was talking about: Google essentially uses the entire population to create massive training sets (which you need for high quality neural nets) and other people use shit like reinforcement learning.
Upper management loves to jerk off about Deep Learning, but unless you can get some interns to do the labeling, they are going to be spending your time (which I assume is somewhat valuable) to essentially be a data entry specialist.

>>60161604
>unless you can get some interns to do the labeling
That may have happened.

Basically, some interns did a summer project which did the NLP procedurally with a moderate success rate that I'm supposed to beat.

If they used any kind of training set then I guess I have that available.

>>60161745
LMAO the classic "Look at what the intern did with R".
You may need to rip that apart, be gentle of course, but I highly doubt they did proper training, testing, validation and all that.
What they most likely did was label data, so get that ofc.

>>60161819
>LMAO the classic "Look at what the intern did with R".
>You may need to rip that apart, be gentle of course, but I highly doubt they did proper training, testing, validation and all that.
>What they most likely did was label data, so get that ofc.

It's a bit like that, they actually wanted the interns to do DL but their supervisor laughed it off and said no way for a summer project (sounds like an awesome supervisor desu).

So now I'm to do the DL 'properly'. My actual boss is probably somewhat on my side here so I shouldn't be too rushed but still...

The interns weren't undergrads though, they were late PhDs so they're probably more qualified than I am, the only experience I have with NLP is once doing something in GATE that didn't work very well. For all I know (right now anyway), that's what the interns did too.

If anyone here knows conv nets, I have a quick question.


Would the data passed from Conv/Pool layer to Fully Connected layer need to be normalized?


Implementing a conv net from scratch, I've been stuck for a couple of weeks, the outputs are always near 0, not sure where to look next.

>>60161946
>Would the data passed from Conv/Pool layer to Fully Connected layer need to be normalized?

Nigger that's like 3 lines of code just try it

Anyway yeah that's gonna give you better results almost every time, but it sounds like your thing is fucked somewhere else

>>60162107
There's a lot of code, and so, places where it can go wrong.

I did try it, but maybe this is fucked as well as something else, and I'd get the same results, I was trying to at least isolate this uncertainty.

Thank you, though. Learning it, bit by bit

>waifu classifier now trained on lewd

it learned to recognize sperm as a sign for probably-NSFW

what the fuck am I even doing

>>60159172
>genetic programming specialist
how to become that?

>>60163799
I did a Masters by coursework in AI (Comp Sci really but took AI as my major stream) and one subject required a project and the lecturer/supervisor gave me a fucking huge task in adding functionality to a GP engine that was kind of his pet project (opensource and technically produced by the faculty but really just him and one other lecturer gave a shit about it).

Anyway, I learned a fuckload about GP and got pretty good at it. Not just at say, implementing a GP algorithm to solve something but at understanding the mechanics of it and what influences code bloat and selection pressure and shit like that. My project was about adding auto-tuning of some of these things to the base engine, utilising covariance of fitness and tuning parameters to determine what was actually contributing fitness and what was coincidental.

Later I got a job in industry doing various shit, only some of which is AI or GP related but since I'm the only guy with AI in my resume, here I am trying to do DL for no particular reason.

>>60156720
>HE FELL FOR THE DEEP LEARNING MEME
>HE ACTUALLY FELL FOR THE DEEP LEARNING MEME

deep learning nigga here, lol ur fucked if you weren't doing this 5 years ago

>>60161946
Yes always normalize everything it makes everything better no matter what :^)

>>60164074
Well, I had specifically meant for the data between those layers. The input data (MNIST-like dataset) is already normalized.

But I've normalized data when using just fc layers, and the data passed to the layers from pool/conv layers is sometimes above/below 1/-1 respectively, so I wondered if maybe this would contribute to the net not getting trained.

But I guess that's maybe what normalization layers are about in alexnet, and they're apparently not that important/impactful

>>60163974
do you mind pointing me to any good GP resources if you have any? i've read a book about it and implemented some stuff in the past, but would like to know it in a more formal way

>>60156720
If that wasn't apparent before you started, there's much bigger gaps in your knowledge you must work on.

>>60164218
>good GP resources
Your best bet is google scholar.

Try these three papers, I learned quite a bit from them. Obviously focused on code bloat because that's what I've researched most thoroughly.
https://krasnow.gmu.edu/socialcomplexity/files/2017/03/A-Comparison-of-Bloat-Control-Methods.pdf
https://www.researchgate.net/profile/Riccardo_Poli/publication/228347495_Covariant_Parsimony_Pressure_for_Genetic_Programming/links/0deec522773ac57d04000000.pdf
http://cswww.essex.ac.uk/staff/rpoli/papers/gptp10_Poli.pdf

>>60164501
>>60164537
thanks a lot

>>60156720

hey op where did you "get up to date on machine learning"

i am looking for good resources but most of what i find is shit

>>60164501 (You)
Oh yeah, and the standard thing to do here when you find a paper that you found interesting or helpful, is to check the references and go read those papers too.
That will give you a good grounding in the seminal papers of the field.

That paper by Poli in particular is fantastic, I can't express how groundbreaking that work was at the time and how much it taught me about the behaviour of populations in the simulation.