We don't need no stinking dictionaries

About: Text Understanding from Scratch - Zhang & LeCun, New York University 2016

From the Conclusion …
 
In this article we provide a first evidence on ConvNets’ applicability to text understanding tasks from scratch, that is, ConvNets do not need any knowledge on the syntactic semantic structure of a language to give good benchmarks text understanding.  This evidence is in contrast with various previous approaches where a dictionary of words is a necessary starting point, and usually structured parsing is hard-wired into the model.

Tags

  1. ai
  2. nlp
  3. neural net
  4. distinctions

Comments


Mark de LA says
Mark de LA 2016-06-09 09:54:54 [item 21009#52722]
Context shifting from machines to humans may be a bit premature. laughing
Seth 2016-06-09 10:22:32 [item 21009#52725]
Well there is a grand urge for our machines to understand us … just so that they can better serve.  When i talk to Siri or Alexa i always tease them because of how stupid they are … i paid good money for these devises, i expect that they should know what i mean when i ask them for something.   Premature?  ← i don’t think so … more like in demand. 
Mark de LA 2016-06-09 10:30:27 [item 21009#52726]
See how long it takes them to care, really care laughing
Seth 2016-06-09 10:48:16 [item 21009#52728]
Well yes of course thumbs up … that is a hard one … even for humans grin
Mark de LA 2016-06-09 10:50:19 [item 21009#52730]
during context-shifting or not … laughing
Seth 2016-06-09 11:17:57 [item 21009#52733]
well the need of “context lifting”  and even of “context shifting” is created by multiple beings having separated their insides from their outsides.  but i don’t get the joke. 
Mark de LA 2016-06-09 11:41:56 [item 21009#52736]
Still assuming your own premise which I don’t accept.  Repetition does nothing towards accepting it either. 
The joke is you are still thinking I accept it & shifting context in the middle of a conversation at the same time.
Seth 2016-06-09 12:54:18 [item 21009#52737]
well certainly it is only my conjecture that a being’s awareness  that its insides are different than its outsides contributes to  its experience of consciousness.  even though, over time i could point out many examples which are constant with that,   i doubt that any kind of “proof” of it will ever happen.   it will be a sythetic affair … not an analytic one.  just maybe look for a jump in intelligence when somebody does in fact get an artificial automated process to be aware of what is part of it, as opposed to what comes from its outside. 

If you actually thought  i expected  you to accept anything that i say at face value, then you are actually dumber than i anticipated.   ← i guess that is why i missed your joke.
Mark de LA 2016-06-09 13:07:10 [item 21009#52738]
Well I guessed you were smarter than you are. I use laughing when it amuses ME! I meet & blog with many people who are barely conscious – just people with jerking knees.  The first problem is knowing what consciousness is not yet clarified by anything you have yet said. 
Seth 2016-06-09 13:19:34 [item 21009#52739]
i guess you didn’t realize all possible interpretations of your clumsily composed sentence,  “The first problem is knowing what consciousness is not yet clarified by anything you have yet said.” … to one of which i heartily agree … Freudian slip? ← grinlaughing
I meet & blog with many people who are barely conscious – just people with jerking knees. ←  grinlaughing

Seth says
Seth 2016-06-09 13:42:10 [item 21009#52742]
anyway getting back to the topic at hand.  

We don’t need no stinking dictionaries to bind word to meaning.  Read the pdf if you doubt, with special attention to astonishing in, “We show that temporal ConvNets can achieve astonishing performance without the knowledge of words, phrases, sentences and any other syntactic or semantic structures with regards to a human language.”   This fear you copped about language usage always just going in circles,  “Znues is Znees” style,  is unfounded and quite misses how intelligence actually binds to what happens. 
Mark de LA 2016-06-09 14:29:57 [item 21009#52746]
Yeah, I don’t agree.  Maybe for those who are machines & have the entire Internet of data to grok & intuit via electronic neural networks it is OK to just Monte Carlo your way to a usage.  It is not important to get the exact meaning of a word.  Google corrects itself with alternative results.  You, for instance don’t much bother with exact meanings of words anyway, often making your own up.  You have no idea what either znuz or znees means except as a title to an autobiography. Awareness = consciousness doesn’t give you what either is – of course being Bozo you could declare something and start a parade, but you might need to look behind you & notice the parade is only one element long. I am more interested in human-human  than machine-human language interfaces. The same goes for being = happening ; whether or not you made that equivalence. To me that is bullshit. The paradigm that keeps you repeating it ad nauseum still seems to me to be one element long .  Any other takers so far?
Seth 2016-06-09 15:20:15 [item 21009#52751]
Well there is no such thing as “the exact meaning of a word” apart from how people have actually used it.  The stastictics of word usage in contexts is the very best “dictionary” that we can consult to determine common meaning. 
Mark de LA 2016-06-09 16:28:36 [item 21009#52754]
So says the guy who could not pass Subject A English at UCLA.  Yep laughing Hard to take you as an authority. 
it’s more of a practical matter, than one of authority.   try it out yourself, don’t take my opinion … try using words to mean just as best as you can what other people who are using them mean …  rather than consulting a dictionary,  and then arguing about their usage which you think contradicts the dictionary.    see if it does not connect you more directly with what you read and write to (and from) those with whom you are communicating.  

also maybe notice that children who learn to speak very well indeed (up to let’s say age 7), never ever consult a dictionary to figure out what somebody means, yet they learn thousands of words and how to use them effectively.    if there are exact words meanings only specified in dictionaries which override how people use words,  then how are the children learning them ?

Seth says
Seth 2016-06-09 13:42:10 [item 21009#52742]
anyway getting back to the topic at hand.  

We don’t need no stinking dictionaries to bind word to meaning.  Read the pdf if you doubt, with special attention to astonishing in, “We show that temporal ConvNets can achieve astonishing performance without the knowledge of words, phrases, sentences and any other syntactic or semantic structures with regards to a human language.”   This fear you copped about language usage always just going in circles,  “Znues is Znees” style,  is unfounded and quite misses how intelligence actually binds to what happens. 
Mark de LA 2016-06-09 14:29:57 [item 21009#52746]
Yeah, I don’t agree.  Maybe for those who are machines & have the entire Internet of data to grok & intuit via electronic neural networks it is OK to just Monte Carlo your way to a usage.  It is not important to get the exact meaning of a word.  Google corrects itself with alternative results.  You, for instance don’t much bother with exact meanings of words anyway, often making your own up.  You have no idea what either znuz or znees means except as a title to an autobiography. Awareness = consciousness doesn’t give you what either is – of course being Bozo you could declare something and start a parade, but you might need to look behind you & notice the parade is only one element long. I am more interested in human-human  than machine-human language interfaces. The same goes for being = happening ; whether or not you made that equivalence. To me that is bullshit. The paradigm that keeps you repeating it ad nauseum still seems to me to be one element long .  Any other takers so far?
Seth 2016-06-09 15:20:15 [item 21009#52751]
Well there is no such thing as “the exact meaning of a word” apart from how people have actually used it.  The stastictics of word usage in contexts is the very best “dictionary” that we can consult to determine common meaning. 
Mark de LA 2016-06-09 16:28:36 [item 21009#52754]
So says the guy who could not pass Subject A English at UCLA.  Yep laughing Hard to take you as an authority. 
Mark de LA 2016-06-09 16:47:51 [item 21009#52756]
Of course your statement puts up for grabs the meaning of the word exact & meaning of statistics relative to a dictionary.  We’ve had usage dictionaries for decades or more & hagglers since the beginning of time – so who knows what you are actually saying – maybe nothing ←   oops that word is very much in dispute. Maybe  I should say some nothing & some not-nothing or not  ponderinglaughing
well i didn't make this stuff up … read into the context given by google language acquisition … or maybe read some korbskinsky semantics … or maybe try to explain Xhang & LeCun’s amazing results … or maybe even try to get a computer to understand you … or maybe watch- a young child learn to understand what you tell them.  

or maybe just stay within your own mind and within the assumptions of those you hold in authority … and bask in that which justifies your hurling uninformed irrelevant insults at my thinking for whatever profit it garners. 

Mark de LA says
Seth 2016-06-11 01:17:35 [item 21009#52767]
Mark: You have no idea what either znuz or znees means except as a title to an autobiography.

Well actually i think i do know what they mean.  GW made up two words and defined each in terms of the other.  It is just GW’s joke.  Nothing more, nothing less.  laughing

Did you get the joke?  I did. 
Mark de LA 2016-06-11 08:32:39 [item 21009#52768]
Turd, here is what exactly happened.  He found the word Znuz  in the Dictionary of American slang and it said “see znees” .  He then looked up znees & found “see znuz”. He showed it to me. The joke is on you. Occasionally in his writing he uses znuz like a synonym of the slang word “cool”.
Seth 2016-06-11 08:39:22 [item 21009#52770]
ok, i thought he made them up … i did not know that he found them in the slang dictionary … which ingorance did not make me a “turd”.   nevertheless it was a joke.   a deeper understanding might be knowing why he chose that joke to title his autobiography.  what is your opinion there?
Mark de LA 2016-06-11 08:49:32 [item 21009#52773]
Maybe after loosing some arrogance as in the title of this item, consult the Akashick Records & find out. Maybe Hotei (Laughing Buddha) is around there somewhere to clue you in. 
laughing
Seth 2016-06-11 08:58:14 [item 21009#52775]
Hotei might justify naming after good humor in general … but i rather think that GW’s choice was much more particular.  but of course i do not know.  Do you?

and incidentally,  me i prefer some arrogance to a wishy washy nothingness … something about the will to be … might you perhaps prefer that i be not?

and incidentally, incidentally,  the thesis that titles this thought is not of my synthesis … that is a thought from others … just not your peculair guys laughing
Mark de LA 2016-06-11 09:14:10 [item 21009#52777]
Arrogance persisting – rwg in full bloom! 
Seth 2016-06-11 09:44:41 [item 21009#52782]
i don’t know if there is a symmetry happening here or not.   i say RWG is just a bad habit, and try to avoid it.  You say it is automatic and do it with zest.  You equate arrogance with a bad habit.  I equate  arrogance with being with zest. 

incidentally i like your image of a rose burning … i get the association of a rose which grows from a manuer pile and the fire which destroys that being here RWG persisting.  Kudos!
?

See Also

  1. Thought Fascinating piece on Internet with 53 viewings related by tag "ai".
  2. Thought Another girl for my nightstand ... with 52 viewings related by tag "ai".
  3. Thought The binary logic of two distinctions with 34 viewings related by tag "distinctions".
  4. Thought Have you ever wondered .. with 29 viewings related by tag "ai".
  5. Thought about: Tutorvista.com - Online Tutoring, Homework Help in Math, Science & English By Expert Tutors with 26 viewings related by tag "ai".
  6. Thought #Field with 26 viewings related by tag "NLP".
  7. Thought Email Chatbots with SparkPost with 25 viewings related by tag "ai".
  8. Thought General overview article ... with 17 viewings related by tag "ai".
  9. Thought about: Intel Launches Movidius Neural Compute Stick: Deep Learning and AI on a $79 USB Stick with 14 viewings related by tag "ai".
  10. Thought [title (23165)] with 12 viewings related by tag "ai".
  11. Thought What is Cybermind ? with 9 viewings related by tag "ai".
  12. Thought Example of online tutoring by computer circa 2015 with 7 viewings related by tag "ai".
  13. Thought about: Semantic Scholar - An academic search engine for scientific articles with 6 viewings related by tag "ai".
  14. Thought about: Thinking Domains as Assistive Technology - comment 61170 with 6 viewings related by tag "ai".
  15. Thought about: A Son’s Race to Give His Dying Father Artificial Immortality with 5 viewings related by tag "ai".
  16. Thought Life is boring repitition ... with 4 viewings related by tag "ai".
  17. Thought Good Girl with 4 viewings related by tag "ai".
  18. Thought Questions - an old NAC (NLP) Applet Design with 3 viewings related by tag "nlp".
  19. Thought Facebook Linguistics with 2 viewings related by tag "ai".
  20. Thought about: using artificial intelligence to influence human behavior with 2 viewings related by tag "ai".
  21. Thought about: Seth's Conjecture with 1 viewings related by tag "ai".
  22. Thought about: google is letting artificial intelligence run search - bloomberg business with 1 viewings related by tag "ai".
  23. Thought about: research blog: inceptionism: going deeper into neural networks with 1 viewings related by tag "ai".
  24. Thought about: jibo - the worlds first social robot with 1 viewings related by tag "ai".
  25. Thought The machines do the translating from Google Blog with 1 viewings related by tag "ai".
  26. Thought about: new mit algorithm rubs shoulders with human intuition in big data analysis - the washington post with 0 viewings related by tag "ai".
  27. Thought about: facebook begins using artificial intelligence to describe photos to blind users | the verge with 0 viewings related by tag "ai".
  28. Thought about: transcendence (2014) - imdb with 0 viewings related by tag "ai".
  29. Thought Deep Mind & Deepnet with 0 viewings related by tag "ai".
  30. Thought about: flp vol. ii table of contents with 0 viewings related by tag "ai".
  31. Thought The Six Blind Elephants Metaphor with 0 viewings related by tag "nlp".
  32. Thought I just noticed Google's recommended stories with 0 viewings related by tag "ai".
  33. Thought Hero Journey - Robert Dilts with 0 viewings related by tag "nlp".
  34. Thought the distinction chair or tree stump with 0 viewings related by tag "distinctions".
  35. Thought about: harvard awarded with 0 viewings related by tag "ai".
  36. Thought Neurons making a distinction with 0 viewings related by tag "ai".
  37. Thought an old ai linguistic buddy with 0 viewings related by tag "ai".
  38. Thought Going Meta with 0 viewings related by tag "nlp".
  39. Thought AI aggregation with 0 viewings related by tag "ai".
  40. Thought about: chainer: a flexible framework of neural networks with 0 viewings related by tag "ai".
  41. Thought a number of ai forms wrapped into one with 0 viewings related by tag "ai".
  42. Thought Lpkb with 0 viewings related by tag "ai".
  43. Thought about: Amazon Mechanical Turk - Welcome with 0 viewings related by tag "ai".
  44. Thought about: wonders what at minimum needs to happen for cyber molecules to circulate freely and stik to each other to meet our intentions with 0 viewings related by tag "ai".
  45. Thought about: ConceptNet with 0 viewings related by tag "ai".
  46. Thought [title (17761)] with 0 viewings related by tag "distinctions".
  47. Thought Machine Translation with 0 viewings related by tag "ai".
  48. Thought Some math musing re philosophy of mind with 0 viewings related by tag "ai".
  49. Thought SeriTD has a message for @seth and @mark. with 0 viewings related by tag "ai".
  50. Thought about: a dialogue ... with 0 viewings related by tag "ai".