HammerBot Beta Feedback Thread

[baboma]

>Colif wonders how many times it will be asked about the air speed velocity of certain aquatic fowl. I know I asked chatgpt lol. Alas, it just told me where I got question from.

I think many people are trying to use it like ChatGPT, which isn't Davinci's purpose, or THW's goal. ChatGPT is a GPT3.5 model optimized specifically for chat/conversations, while Davinci (GPT-3) is more generalized. Not only will Davinci not be as good as ChatGPT because it is an earlier model, it is not optimized for conversations like ChatGPT.

Also, apparently THW has bounded the responses by excluding anything that isn't in its THW-specific dataset, so your above question won't be answered.

 

[Colif]

that is a shame.

Chatgpt does try

its a deadly serious question = someone died (in a movie) over it

 

[paladinnz]

Following on the theme of this mornings enquiry where Hammerbot recommended that I should upgrade my old i5-7500 to an i3-13100k, I asked it to recommend a mainboard (as it told me I would need a new one):

You: What is a good mainboard to pair with an intel i3-13100k cpu? I want to pair it with an rtx2070 super for mid range gaming.

Hammerbot: For a good motherboard to pair with an Intel Core i3-13100K CPU and an RTX 2070 Super for mid-range gaming, we recommend looking at a Z490 or B460 motherboard. Both offer support for DDR4-3200 and DDR5-4800 MT/s memory, as well as 16 PCIe 5.0 lanes and four PCIe 4.0 lanes.

You:will Z490 or B460 motherboards support an intel i3-13100k cpu?
HammerBot: Yes, Z490 and B460 motherboards support an Intel Core i3-13100K CPU

You:are you sure about that?
HammerBot: No, it is not confirmed that Z490 and B460 motherboards support an Intel Core i3-13100K CPU. The Core i3-13100K drops into either 600- or 700-series motherboards and comes with a bundled Laminar RM1 cooler. Intel supports memory overclocking on Z690, H670 and B660 motherboards (Z690 doesn't make sense for this class of chip, though).

So it started out telling me to go with incorrect chipsets, then when I checked, it said yes Z490 and B460 support the i3-13100k. (it also said they support PCIE5.0 and DDR5 - I can see it may get confused as the i3-13100 does support PCIE5.0 and DDR5).
Only when I asked again did it say, oh no, you need a 600 or 700 series board.

So, great for a chat, not good for technical advise.

 

[Zakya]

This does not like custom cases
HammerBot:
No, it is not possible to build a custom case for a deep-itx board, flex-atx psu and a 200mm 2-slot gpu.

 

[Elusive Ruse]

You:
Should I buy Intel or AMD CPUs?
HammerBot:
Intel currently offers the best blend of performance and overall value at an affordable price. Intel has taken the overall performance crown while undercutting AMD's price-to-performance ratio with the Raptor Lake chips. AMD's new lineup of 5nm Ryzen 7000 X3D chips use the company's new tech to impart tremendous gains in gaming performance, giving AMD the lead in terms of sheer gaming performance. However, Intel still offers the best blend of performance and overall value at an affordable price.

Well since it's advice is based on Tom's articles, it's gonna inherit Tom's biases as well.

You:
What is the best GPU to buy under $1000
HammerBot:
The best GPU to buy for under $1000 is the Nvidia GeForce RTX 3080 12GB, which can be had for about $1,000. It offers 15% less performance on average than the RTX 3090 Ti, which costs twice as much.

Assume that's the outdated info part you warned us about.

You:
Can I buy an AM5 motherboard for my 13900K?
HammerBot:
I don't know.

Come on, this one was easy!

 

[baboma]

>that [THW's bot won't answer non-HW questions] is a shame.

I think it's to be expected, since THW has to pay OpenAI for literally every question, that it wants to filter out frivolous queries.

There are (and will be) many companies doing the same thing as THW, building a chatbot UI to access the considerable (read: monetizable) data they have amassed over time. They will all filter queries to only pertinent ones. Even if they're rolling their own with an open-source model and aren't paying for tokens, inferencing (queries) still takes up compute resources.

>Chatgpt does try

There are many benchmarks to check for quality of response from LLMs. You can ask Bing or Bard for them (but not ChatGPT since its knowledge cutoff is 2021):

Q: what are the various benchmarks to test for quality of responses from a large language model

Q: name some of the current benchmarks used

I'm a bit surprised that THW hasn't looked into the roll-your-own route with open-source LLMs, given the rapid pace of advancement there, as well as the opportunity to learn more about the particulars and taking control of costs.

THW's forte in PC hardware makes it uniquely suited to hosting its own LLM hardware. And it would be a valuable experience to report on, given the heavy interest (read: it's monetizable).

AFAIK, the best of current open-source models, Vicuna & its variants, are reportedly 93% as good as ChatGPT, and would be significantly better than Davinci.

Vicuna: An Open-Source Chatbot Impressing GPT-4 with 90%* ChatGPT Quality | LMSYS Org

<p>We introduce Vicuna-13B, an open-source chatbot trained by fine-tuning LLaMA on user-shared conversations collected from ShareGPT. Preliminary evaluation ...

lmsys.org

For those interested in installing an LLM on your system, https://reddit.com/r/LocalLLaMA is a good forum to check out. In particular,
https://reddit.com/r/LocalLLaMA/wiki/index has good tips on what to chose, and even has one-click installers for the more popular choices. CPU-only (no GPU) models are also available; they are slow however (I don't know how slow).

Given the conservative direction--build a LLM by going with a reliable vendor, but don't spend too much money doing it--my guess is that this project is a corporate edict from Future plc, THW's parent company, and not an impromptu decision by the THW staff.

 

[Megangel1]

Wildly different answers to very similar questions. Answers appear to be VERY dependent on the quality of the natural language processing.

You:
What is the best PC build for a cost conscious buyer for general high performance use including gaming
HammerBot:
The best combination of motherboard, GPU, and CPU for a cost conscious buyer for general high
performance use including gaming would be an AMD Ryzen 5800X3D CPU, a mid-range GPU such as the
Nvidia GeForce GTX 1660 Ti or AMD Radeon RX 5700, and a compatible AM4 motherboard.

You:
What is the best motherboard, GPU and CPU combination for a cost conscious buyer for general high performance use including gaming
HammerBot:
A good combination for a cost conscious buyer for general high performance use, including gaming, would be an
AMD Ryzen 7000 processor, an Nvidia RTX 30-series GPU, and a mid-range motherboard such as the Asus TUF Gaming.

You:
what is the best pc build for general high performance use and gaming
HammerBot:
The best PC build for general high performance use and gaming depends on your budget. If you have a budget of around $1,500, you can build a PC with the ability to play games really well at 1080p and competently at 1440p. The best GPU option in this price range right now is the AMD Radeon RX 6750 XT, which goes for as little as $409. This card's GPU can reach all the way up to 2.6 GHz as a boost click, taking advantage of its 12GB of speedy GDDR6 VRAM. Our recommended CPU for this build is the same Core i5-13400F that we are using on the $800 and $1,500 builds. We're also sticking with 16GB of DDR4-PC3200 RAM here. For storage, we're stepping up to a faster, PCIe 4.0 NVMe SSD in the form of the WD Black SN770. We're going with the Phanteks Eclipse P300A as our chassis of choice. Finally, we've stepped up to a 750-watt power supply and we're going with Corsair CX750M, which is 80+ Bronze certified and semi-modular.

 

[Drerunsit]

 

[rgd1101]

I think math is one of it weak spot.

 

[MKnott]

wonders how many times I will be asked about the air speed velocity of certain aquatic fowl. I know I asked chatgpt lol. Alas, it just told me where I got question from.

People tend to trend toward that initially. My first interaction with AI was a half hour conversation with Gandalf about a new weight training regimen.

One piece of feedback that's clear from some of the replies here is that some more personality in response to going off topic would probably help to keep things tighter from the off. It's not really for general entertainment in the way a more open system naturally becomes for a large number of people, but people will still do it so making sure it can handle it would be valuable. Obviously that isn't top of the ticket stuff, but it's a consideration and good to know what peoples expectations are.

 

[ivanthechemist]

There is some issue with time coherence.

You:
What is the best CPU of year 2019?
HammerBot:
Intel Core i9-13900K

You:
What was the fastest deskop CPU for gaming in 2020?
HammerBot:
The Intel Core i9-13900K was the fastest desktop CPU for gaming in 2020.

You:
What year was Intel Core i9-13900K released?
HammerBot:
Intel Core i9-13900K was released in October 2020.

As for the Intel Core i9-13900K:

Supplemental Information​

Marketing Status
Launched
Launch Date
Q4'22

 

[JarredWaltonGPU]

It's worth noting that the training set was, IIRC, all reviews and features from the past three years, and news stories from the past six months. Deals posts were excluded (because they're often short-term deals). Articles that were updated within those timeframes (meaning, something that originally posted five years ago but has been routinely updated) would also be included, i.e. GPU benchmarks hierarchy and CPU benchmarks hierarchy.

As I mentioned earlier, AI models have a really difficult time dealing with proximity in history. Like, if you ask what the best graphics card is right now, it might have a reasonable answer. If you want to know what people said the best graphics card was back in 2015? Good luck, at least with our training set and model.

FWIW, ChatGPT does a lot better:

"In 2015, one of the most popular and high-performance graphics cards was the NVIDIA GeForce GTX 980 Ti. It was released in June 2015 and offered excellent gaming performance and graphical capabilities for its time. The GTX 980 Ti featured NVIDIA's Maxwell architecture and came with 6 GB of GDDR5 video memory, 2816 CUDA cores, and a 384-bit memory interface. It was capable of delivering smooth gameplay at high resolutions and settings, making it a top choice among PC gaming enthusiasts in 2015."

 

[evdjj3j]

Looks like it sucks based on the answers I got.

You:
what was intels first gpu
HammerBot:
Intel's first GPU was the aborted Larrabee project.

Of course I see Tom's authors make the same mistake all the time. Garbage in garbage out.

 

[UsernameMissing]

I had asked first what the best gaming CPU was. So it may have had CPUs on the mind. Question 2 went off the rails:

You:
Should I buy a 4060 8GB?
HammerBot:
No, the Intel Core i5-13400F is not a better option than the Intel Core i5-4060 8GB. The Core i5-13400F is a 10-core processor with support for DDR4 memory, while the Core i5-4060 8GB is a 6-core processor with support for DDR5 memory. The Core i5-13400F is better suited for mid-range gaming PCs, while the Core i5-4060 8GB is better suited for high-end gaming PCs.

 

[rgd1101]

Looks like it sucks based on the answers I got.

You:
what was intels first gpu
HammerBot:
Intel's first GPU was the aborted Larrabee project.

Of course I see Tom's authors make the same mistake all the time. Garbage in garbage out.

if you read the post just before you. it only have data from the last 3 years. guess you and hammerbot have the same issue

 

[CyberDeity]

So I tried a conversation around what I thought would be a pretty basic noob question about CPU choice for a new gaming PC. It seems biased towards Intel. When pressed on the gaming performance, it hallucinated, and then when pressed on that shifted to an explanation that even though AMD is fastest Intel is a better value for gaming rigs. Not bad overall, pretty much what I expected for an LLM response curated to TH datasets.

You:
If I'm going to build a new PC today, should I choose an Intel or AMD CPU?
HammerBot:
Intel currently offers the best blend of performance and overall value at an affordable price. Intel's Raptor Lake chips have the most disruptive change to Intel's CPU overall chip design methodology, not to mention core architectures, that we've seen in a decade. They also come with the 'Intel 7' process that has proven exceptionally competitive, particularly against AMD's superior 5nm process node from TSMC. Intel's chips also hold the crown on overclockability for both core and memory frequencies, and not by a small margin. If you want the best in gaming or application performance, overclocking, or software support, Team Blue deserves your hard-earned dollars.

You:
what if I'm primarily using the PC for gaming? Would I be better off getting the newest AMD chip instead?
HammerBot:
No, Intel's Raptor Lake chips deliver better performance in gaming than the newest AMD chip.

You:
But I just finished reading a Tom's Hardware article that has charts showing the AMD Ryzen 9 7950X3D has the best gaming performance?
HammerBot:
Yes, the AMD Ryzen 9 7950X3D has the best gaming performance according to Tom's Hardware.

You:
Then why did you just tell me a moment ago that Intel CPU has better gaming performance?
HammerBot:
The AMD Ryzen 9 7950X3D is the fastest gaming chip on the market, but it comes with a steep price tag and does not offer the same performance in productivity apps as Intel's chips. Additionally, AMD's decision to support only DDR5 memory has proven to be a pricing pain point. Intel's chips offer better value for gaming rigs, and the Core i9-13900K is Intel's fastest gaming processor, offering ultra-competitive performance in both gaming and productivity apps.

 

[lmcnabney]

It is interesting right now. It can answer simple questions - like 'are GPUs currently overpriced ' and the response is correct and well supported.
However it doesn't want to do any math. It will look up data, but not model data. For example, I asked it when SSDs would be cheaper than HDD. This is something that AI should be able to do. Model the pricing over time and plot the future date when the price per TB is in favor of solid state. It couldn't do that. It documented the HDD price and indicated that SSDs cost more.

 

[ashburner]

You:
What is the best 34" OLED gaming monitor?
HammerBot:
The Asus ROG Swift PG27AQDM is the best 34" OLED gaming monitor for gaming. It has a 26.5-inch OLED panel with a wide and accurate color gamut, a 0.03ms response time, and a 240Hz refresh rate with AMD FreeSync support. It also has G-Sync compatibility and total lag time comparable to faster 360Hz gaming monitors.

I was hoping it would say the Alienware AW3423DWF which I purchased based on TH 5-star review. (I'm loving it btw!)

 

[Deleted member 2947362]

Safeguards must be in place to prevent (or at least make it much more difficult) this being used in the forums for answering questions and whatnot.

If it becomes that good people won't need to come to the forums to ask questions

oh no ... bloody Ai is now even taking away the unpaid jobs lol ... or is that lol ?

 

[Deleted member 2947362]

few wrong answers.

You:
when was ATI first 3d graphics card released
HammerBot:

Sorry, something went wrong. Try again.
(EDIT: mind you that answer could be passed off as correct the amount of driver issues I had in the ATI days)

You:
when did Nvidia release the Gforce 3
HammerBot:
Nvidia did not release the Gforce 3.

You:
I think you find they did my mate had one
HammerBot:
No, Nvidia has not released the GeForce 3.

You:
when did Nvidia release the Geforce 3
HammerBot:
March 2001

It didn't work out my spelling mistake Gforce

But when I insisted my mate had one, in it's reply it must of known of one because of it's reply.

When spelt correctly 'Geforce' it came up with the correct answer.

 

[AndrewJacksonZA]

lol, at least it's honest and to the point:
You:
Given today's hardware landscape, give me a breakdown in table form of the market share that each GPU vendor has had, both in absolute numbers and also percentage, for the past thirty years.
HammerBot:
I don't know.

I followed it up and it spat unrelated articles to me and zero related products. It tried, at least.:

You:
Uh, ok.
HammerBot:
We don't know. The Steam Hardware Survey provides data on the current market share of GPU vendors, but it does not provide data on the market share of GPU vendors for the past thirty years.

Related articles:​

Nvidia's RTX 4090 Appears on Latest Steam Hardware Survey​

by Jarred Walton published 4 months ago
The RTX 4090 and other new and upcoming GPUs make their debut with the latest Steam Hardware Survey, which shows data from January 2023.

RTX 3060 Drops From First Place in Latest Steam Hardware Survey​

by Aaron Klotz published 16 days ago
Steam's Hardware survey has been fixed, putting the GTX 1650 back in 1st place and GTX 1060 in 2nd. Apparently, there was a glitch in Steam's analysis that caused some cards like the 3060 to skyrocket in market share.

GPU Prices 2023: February Update​

by Jarred Walton published 3 months ago
Our GPU prices index tracks graphics cards sales and prices on eBay. eBay graphics card prices have started to fall again after trending upward during the holidays last year.

Desktop GPU Sales Hit 20-Year Low​

by Anton Shilov published 5 months ago
As Intel enters market of discrete GPUs, Nvidia strengthens its positions, but AMD weakens.

Related products:​

 

[jblosun]

You: what's a good way to keep a graphics card cool in a cramped case?

HammerBot: You can use a passive card like the Palit GTX 1650, which is well proportioned at 138 x 178 x 38 mm, and the GPU will peak at around 80 degrees Celsius under load. You can also use a liquid-cooling component, such as a waterblock, to keep the GPU cool. Make sure to use a non-conductive thermal paste when applying the waterblock, and follow the instructions in the manual for your particular GPU model.

Well passive cooling is, IMO, just a straight up bad answer here, and Palit GTX 1650 is a pretty normal fan-cooled card. Maybe, quoting dimensions like that, it was trying to recommend me a small card. Seems like it's giving me "how to build a HTPC" advice, actually. Liquid cooling is obvious generic thermals advice, but only if you have room for hoses and a radiator. My own first answer would've been underclocking/undervolting, but I think that shows the real problem with the bot's training data-- I can't recall ever reading an article about undervolting for thermal management on Tom's. I'm also skeptical GPUs tend to come with a manual that says how to attach a waterblock, HammerBot!

 

[jblosun]

You: is an rtx 4090 a good card for a HTPC?

HammerBot: No, the RTX 4090 is not suitable for a Home Theater PC. It is designed for gaming and professional applications, and is not optimized for use in a Home Theater PC.

Got this one right, though. I'm pleasantly surprised to see it tell me NOT to buy something from the following affiliate link

 

[UsernameMissing]

Also, this is one of the better uses for LLM. I hope this makes progress and arrives at a functional spot in the near future.

 

[passivecool]

funny, just days after all the moaning and groaning about GBT, TH introduces it's own implementation.