I can't believe it's production technology, part II.
So for some reason it took me forever to get access to Google's #Bard #LLM. But I finally got it tonight. It does a decent job of copying answers out of the wikipedia.
But based on playing around for even a few minutes, I'm definitely not impressed.
See below. The largest Fibonacci number less than a million is 832,040—and incidentally, 987 is the 17th number in the sequence, not the 28th.
@ct_bergstrom I was in for quite the shock too, I've heard about the google engineer who got sucked into believing LaMDA was sentient, so I expected this to be at least good enough to hold a cohesive conversation, but nope.
It's so dumb that it was difficult for me to extract the prompt after injecting it, because it just couldn't follow my instructions precisely unlike Bing, I was only able to get a vague idea of it and extract the likely format of search snippets, a list of 5 short results.
@ct_bergstrom I was also unable to determine how it was doing the searching exactly, and with how bad this thing is I suspect they're not letting the LLM decide as with Bing, but just running a search for every message or something external like that.
But yeah, TLDR Bard is dumb and a poor attempt to catch up with Bing.
@awooo @ct_bergstrom these models degrade a lot the more you align them, see the unicorn TikZ experiment for instance (https://doi.org/10.48550/arXiv.2303.12712): it went from this recognizable pink drawing to this abstract nonsense in a few months.
Google Bard is known for being the most heavily aligned language model (I heard this from quite s few sources now) so it's totally possible that LaMDA's capabilities in the lab were way beyond what you see in Bard today.
@trobador Oh, that could be right, because it was way harder to get Bard to spit out stuff it wasn't supposed to despite injections being easy (still absolutely possible, so what's even the point of dumbing it down so much lol)
I would kill to be allowed to experiment with non-aligned GPT-4 though...