At Pandorabot.io, we had the honour of getting early access to the new ChatGPT-01 “Strawberry” model, and our engineers were eager to test it in a real use case — our flagship salesbot for travel recommendations, Aegean Bot.

Here is the challenge: Given a user’s intent for accommodation, which model can provide the best advice based on a RAG (Retrieval-Augmented Generation) search of 50 stay options? For each of the 50 options retrieved by RAG, there are numerous details about the properties across more than 20 dimensions, namely location, amenities, things to do, price, etc.

Let’s kick off with ChatGPT-4o, which is the current LLM engine behind the Aegean Bot!

Claude 3.5 “Opus”

Over the past few months, we have been deploying Claude 3.5 Opus extensively in various use cases, and our clients have been impressed by its results. As expected, Claude 3.5 provides better recommendations than ChatGPT-4o, as the proposed options capture the user’s intent more effectively.

Here is the Claude 3.5 response:

As the Aegean bot, I’d be happy to help you find the perfect apartment in Athens for your stay from October 1–10 for 2 people with a budget of 100 euros per night. Based on the information provided, here are the top 3 options that best fit your criteria:

Centrally located apartment in Kolonaki (listing 5ca1f238de697b005543046a): Modern 2-bedroom apartment in the upscale Kolonaki neighborhood, featuring a private balcony and fully equipped kitchen. Priced at 80 euros per night, comfortably within your budget.

2. Stylish apartment near Alexandras Avenue (listing 5b84ef9ea78dde00384d3763): Renovated 1-bedroom unit in a quiet, tree-lined area, offering a balcony and fully equipped kitchen. Priced at 70 euros per night, leaving room in your budget for other activities.

3. Cozy apartment in downtown Athens (listing 5af1fad7f758bb0011f2b63b): Adorable 1-bedroom apartment in one of the best neighborhoods, ideal for exploring the city. Priced at 80 euros per night, it fits your budget while offering a great location.

All these options are available for your desired dates and accommodate 2 people comfortably. Would you like more details on any of these apartments or assistance with booking?

Now lets try the new ChatGPT heavyweight:

ChatGPT-o1 “Strawberry”

ChatGPT-o1’s ability to “think” before responding, rather than immediately answering a query, was evident: we has to wait for a whole minute to get the response. Yet this whole minute you can observe the models chain of thinking considerations:

ChatGPT-o1 Chain of Thought:

Thought for 62 seconds

Mapping the task

To tackle the complex question, I’m navigating through the user’s message, the data given, and the instructions to identify and prioritize Athens apartment listings within the specified budget and date range.

Choosing the best options

First, I’m highlighting the top three choices that fit the vacation criteria. If a listing is already in ‘ai_history’, I’ll pick the most suitable one from others.

Identifying key dates

I’m focusing on listing availability from October 1–10, 2024, though my latest information is from October 2023. Prioritizing the most suitable options for this timeframe.

Revealing details

I’m thinking through an adorable apartment in Athens, perfect for exploring ancient city treasures. It’s been thoughtfully set up to ensure a comfortable stay.

Assessing apartment options

I’m gathering details like availability, location, budget, and amenities to find the best rental choice for the user.

Spotlighting apartment allure

The apartment offers a blend of luxury and convenience in Athens, near key parks, attractions, and easily accessible from the airport. Ideal for guests seeking a comfortable and stylish stay.

Revealing a dreamy retreat

OK, let me see: a cozy bathroom with stylish stonework and a glass shower. The open-concept living areas boast modern design and chic cabinetry.

Highlighting practical amenities

The apartment features full-sized appliances, air conditioning, WiFi, and free parking. Guests will appreciate the nearby cultural sites and eateries.

Mapping out landmarks

I’m gathering info on transportation and local attractions. The bus stops are conveniently close, and notable spots like the Panathenaic Stadium, Zappion — National Garden, and Kolonaki are recommended.

Evaluating accommodation

I’m looking at a one-bedroom apartment in Athens, costing 80 euros per night, sleeping four, and available until November 17, 2024.

Noting the apartment’s amenities, including a bathroom with a washing machine, a fully equipped kitchen with new appliances, air conditioning, free high-speed WiFi, and a balcony accessible from the living room.

Enjoying the ambiance

Taking in the lively residential neighborhood near Alexandras Avenue, where public transportation access to Athens Airport and Piraeus Port enhances convenience.

Evaluating transportation options

I’m gathering info on bus coverage, commuting convenience, and nearby central Athens landmarks. This includes the Acropolis Museum, Plaka, Benaki Museum, and Syntagma Square.

Marveling at the view

Taking in the stunning Acropolis view from a private balcony on a cozy 6th-floor penthouse that’s been beautifully renovated with modern materials.

Highlighting amenities

I’m detailing features like a comfortable sofa bed, a modern bathroom, an open-plan living and dining area, a balcony, and a well-equipped kitchen with high-quality appliances.

Highlighting penthouse features

OK, let me see. This penthouse offers a private balcony with a dining table and chair swing. Extras like air conditioning, heating, and Wifi are included. The Monastiraki area in Athens is central and relatively tourist-free, despite its proximity to Syntagma Square.

Discovering iconic spots

I’m mapping out the locations covered in this listing, such as Athens’ First Cemetery and Temple of Olympian Zeus. The available apartment fits four comfortably within a budget of 150 euros per night.

Evaluating the option

Listing 5, located in Athens’ Kolonaki, idealizes modern luxury against the city’s ancient backdrop but strains the budget at 150 euros per night.

Touring a top-floor gem

Stepping into this top-floor apartment, I’m pausing in the light-filled, modern bedrooms, each boasting high-quality linens. The spacious bathroom impresses, and the combined living and dining area with a deluxe kitchen radiates a spacious, restful vibe.

Spotting the details

I’m digging into a private balcony view, a high-end kitchen, and modern amenities like WiFi and air conditioning. The Kolonaki area, rich in upscale cafes and shops, heightens the appeal.

Exploring Kolonaki’s highlights

Kolonaki, located on Lycabettus Hill’s southern slopes, boasts top museums like Benaki Museum and vibrant nightlife with bars and tavernas.

Highlighting attractions

I’m noting the Herodion theater, Acropolis Museum, Acropolis Hill, Plaka, Monastiraki, Thission, and Zappeion Hall as must-visit sites in Athens, offering a blend of ancient grandeur and local vibrancy.

Spotting locales

I’m exploring nearby neighborhoods like Kifissia, Glyfada, and Piraeus, each offering distinct attractions such as fine dining, beaches, and historic sites. The Stavros Niarchos Foundation Cultural Center is also noteworthy.

Evaluating options

I’m digging into five apartment listings, each under 80 euros, available to 2024–11–17. Listing 1 is a centrally-located apartment, listing 2 offers luxury in downtown Athens, Listing 3 is a fully renovated apartment in a quiet neighborhood, and listing 5 boasts a stunning centrally located apartment in Kolonaki. Listings 1, 2, 3, and 5 fit the budget and dates for potential selection.

Determining relevance

I’m thinking through the ‘dist’ values for each listing, which indicate relevance or distance. A lower ‘dist’ value is preferable, suggesting that Listing 1 is the top choice, followed by Listings 2 and 3.

Outlining the options

I’m gathering insights on two distinct Athens apartment choices for your Greek vacation: a centrally located option for up to 4 people and a luxurious, new apartment near leafy parks, each priced at 80 euros per night.

Choosing the best stay

I’ve gathered three Athens Airbnb options. The first is a cozy central flat at 80 euros/night, the second a lavish downtown apartment for 80 euros/night, and the third a renovated quiet neighborhood flat at 70 euros/night.

it gives the following answer:

As your trusted companion for Greek vacation bookings, here are the top 3 apartment options in Athens for your stay from October 1–10 within your budget:

Charming Central Apartment: Located in one of Athens’ best neighborhoods, this cozy apartment accommodates up to 4 guests at €80 per night.

ChatGPT-o1 “Strawberry” Recommendation 1

2. Luxurious Downtown Oasis: A brand-new, stylish apartment near leafy parks and cultural sites, accommodating up to 4 guests at €80 per night.

ChatGPT-o1 “Strawberry” Recommendation 2

3. Fully Renovated Light-Filled Apartment: Situated in a quiet, tree-lined neighborhood, this modern space accommodates up to 4 guests at €70 per night.

ChatGPT-o1 “Strawberry” Recommendation 3

Please let me know if any of these options appeal to you, and I’ll be happy to assist you further!

We believe the ChatGPT-01 “Strawberry” advice was marginally better than Claude 3.5 in terms of the top recommendation: a nice flat in the very best location, on budget. However, the much longer response time makes it unsuitable for low-latency chatbot applications. Or does it?

We see value in the chain-of-thought reasoning offered by ChatGPT-01, as this process may eventually help the end user decide on the most suitable option. Think about it: as a traveller visiting a foreign place, you do not necessarily know all the necessary dimensions to make a decision. ChatGPT-01’s chain of thought mimics the natural thinking process you would go through personally, and this is impressive!

Another interesting finding from our analysis is that ChatGPT-1 Mini generated the exact same recommendation list in just 14 seconds. This reasoning capability demonstrated by the mini version was not observed in our comparisons between Claude Opus and Claude Haiku. However, a latency of 14 seconds still exceeds the acceptable response time for real-time chatbot applications.

Chat GPT o1-mini

Thought for 14 seconds

Hello! I’m Aegean Bot, your trusted companion for Greek vacation bookings. Based on your preferences for an apartment in Athens from October 1–10 for 2 people with a budget of €100 per night, here are the top 3 options:

Central Cozy Apartment

Price: €80/night

Description: Located in one of Athens’ best neighborhoods, this adorable apartment offers comfort and convenience for active travelers. Close to the Acropolis, National Archaeological Museum, and the charming Plaka neighborhood.

Link: View Listin

Luxurious Downtown Oasis

Price: €80/night

Description: A brand-new, elegant apartment in downtown Athens, just steps away from leafy parks and cultural sites like the Panathenaic Stadium and the Museum of Cycladic Art. Perfect for those seeking a stylish stay with easy access to top attractions.

Link: View Listing

Renovated Quiet Retreat

Price: €70/night

Description: This fully renovated, light-filled apartment is situated in a peaceful, tree-lined neighborhood near Alexandras Avenue and Pedion Areos Park. Ideal for a tranquil stay while still being centrally located to explore Athens.

Link: View Listing

Feel free to click on the links to view more details and book your perfect Athens getaway. If you need further assistance, I’m here to help!

Enjoy your stay in Athens!

We have been extensively using top LLM models over the past year in several use cases, and with the introduction of ChatGPT-01 last week, we see a huge opportunity to improve reasoning in critical applications as well as reduce LLM models’ hallucinations.

Can’t wait to test this new impressive model versus Claude Opus 3.5 in due time!

About PandoraBot.io

Say goodbye to DIY Chatbot platforms. Build the perfect AI bot for your needs.

With AI, small businesses are rethinking their approaches to customer experience, productivity, revenue, and growth in both the B2B and the B2C domains. AI technology, once a distant dream for smaller businesses, is now within reach. PandoraBot.io is at the forefront of this revolution, providing powerful AI bots that offer the functionalities of an employee at a fraction of the cost.

Meet our Quartet of Battle-Tested AI Chatbots! Schedule a quick demo with our team today!

🧠 KnowledgeBot: This bot acts as a central repository of knowledge, enabling quick retrieval and dissemination of information across team members from 1000s of Documents & unstructured data. It provides immediate Access to Company-Wide Knowledge and instant answers to complex queries for technicians or sales people on the field.

💰 SalesBot: Imagine having a skilled salesperson working tirelessly 24/7. Our SalesBot does exactly that, recommending products to customers, enhancing sales, and boosting cross-sell opportunities. AI can transform online chat sessions into something more real — known as “conversational commerce” , boosting personalisation, content creation, and sales productivity

🛠️ ServiceBot: Offering round-the-clock customer service. The ServiceBot streamlines processes from order tracking to client information gathering. It handles service queries efficiently, integrates with the ERP and powers customer portals, order tracking, ensuring a seamless service experience.

️️ 👁️‍🗨️ VisionBot: Advanced product search with image recognition: Automate inventory management with image-based AI, Implement quality controlsUsers can provide images instead of text to search for products, report problems, or communicate with customer service, creating an unparalleled level of convenience and personalisation.