The Edge Runner Agent #Issue 1
Welcome to the first post of the “Edge runner Agent” series 🎉🤖
If you’re here, it means you’re interested in learning about AI and computer science. And guess what? Me too! 🙋♂️ Who am I? Just another curious person like you that loves computers and talking about AI, and science in general.
Now, you might be thinking, “Great, another series about AI and computer science. What makes this one different?” 🤔 Well, the main difference is that I’m not an expert. I’m not a PhD scientist or a genius who understands everything instantly. I’m just a regular person who is passionate about this stuff and wants to share thoughts with others. 💬
The value of this series is that I’ll try to explain all of these new technologies in simple, easy-to-understand language for both technical and non-technical readers.
I’ll be breaking down complex papers and concepts related to AI and computer science so everyone can understand them.
So, why write a series of articles? Well, I like to take notes on everything I learn or research on a daily basis.
Writing articles is a way for me to pull all those pieces together and organize my thoughts on a particular topic.
Plus, with all the new techniques and research that have come out in recent years (or even months!), I feel like I need to keep up to date and find some order in all this exciting but overwhelming information. 🤯
The edge-runner agent series
A little bit of context about this series of articles: the name, ‘Edgerunner,’ was inspired by the Netflix series of the same name. Upon researching the definition of the title, I realized it was closely related to the topics I wanted to cover in these articles. Essentially, Edgerunner tells the story of a street kid trying to survive in a technology and body modification-obsessed city of the future.
Like the show, this series will explore cutting-edge AI and computer science. 🚀
So long story short, this series aims to clarify the overwhelming amount of information about AI and computer science we encounter daily as technology progresses faster than our ability to fully understand it.
Each month, I’ll be sharing the latest and greatest in the world of AI and computer science 🚀 I’ll also throw in some classic but still interesting topics from time to time 🤓 Here’s what you can expect in each article:
- 🏆 The Edge-Runner Topic: In this section, I’ll break down the most buzzworthy AI or computer science topic into simple, easy-to-understand language.
- 📚 Literature lab: In this section, I’ll take a deep dive into a complex paper or article and break it down into more digestible pieces.
- 💬 Sides discussions: In the final section, I’ll discuss a related topic in AI or computer science.
Let’s begin with the #issue1
If you’re not familiar with ChatGPT, don’t worry! 😎 I’ll introduce you to this amazing tool developed by OpenAI. ChatGPT is a really cool tool that allows you to communicate with AI in a natural, conversational way.
Released freely for research purposes on November 30, 2022, ChatGPT made a big impact on the tech and non-tech community with its astonishing potential for many fields, such as answering questions, admitting its mistakes, challenging incorrect premises, and rejecting inappropriate ones. It was a huge hit, and everyone was talking about it. ChatGPT quickly became a go-to tool for those interested in AI and its potential applications.
What sets ChatGPT apart from other chatbots is its ability to understand the context and generate appropriate responses.
For example, if you ask ChatGPT about the weather in New York, it will be able to understand that you are talking about a city in the United States, rather than a city in another country with the same name. This level of understanding allows ChatGPT to have more meaningful, accurate conversations with users.
Alright, let’s dive into this awesome technology! 🤖 The best way to get started is by asking ChatGPT what it is 🤗. So, let’s give it a try:
How does ChatGPT work?
ChatGPT uses machine learning and natural language processing (NLP) techniques to generate text based on the context of a conversation. When you interact with ChatGPT, you can enter a text message or prompt, and the chatbot will generate a response based on its pre-trained knowledge of language patterns and structure.
What are the potential uses for ChatGPT in non-tech fields?
While ChatGPT is often used in tech-related fields, it has the potential to be useful in a wide range of non-tech fields as well. Here are just a few examples:
- 💬Customer service: ChatGPT can handle customer inquiries and support requests, providing quick and accurate responses to common questions and issues. It could become really handy for Fintech services where the knowledge area is hard to understand, and the customer will need help most of the time to understand what’s going on
- 🏥 Healthcare: ChatGPT could be used to create virtual assistants that can provide patient information and support, such as answering questions about medications or managing appointments.
Wrapping up
Overall, ChatGPT is a powerful tool for anyone looking to have engaging, natural conversations with a chatbot. Another topic would be the ethical concerns surrounding ChatGPT’s use and impact (might be an interesting topic for further research in future articles). For example, there is a risk that ChatGPT could be used to spread misinformation or manipulate public opinion. It’s important to carefully consider these ethical implications and take steps to ensure that ChatGPT is used responsibly.
Give it a try and see how advanced language processing technology can enhance your chatbot experience!
Another exciting AI milestone at the end of 2022 was the achievement of human-level play in the game of Diplomacy by CICERO 🎉🤖 While it may have flown under the radar due to other things such as ChatGPT or Stable Diffusion, this achievement represents a major breakthrough in cooperative AI 🙌 That’s why I wanted to discuss it as my first paper review of the year 📄
🏆 Key takeaways:
- CICERO is a powerful AI that achieved human-level play in the game of Diplomacy. A major breakthrough in cooperative AI
- This achievement demonstrates the potential for combining language models with strategic reasoning in AI and opens a new branch of ethical considerations.
A Closer Look at CICERO
One of the major long-term goals in the field of AI is to build agents that can communicate intentionally with humans using language within interactive environments while understanding the goals and intentions of human behavior. But when it comes to effective negotiation, persuasion, and building relationships with other humans, the challenge becomes even more complex. This is where CICERO excels in the game of Diplomacy.
Why the game Diplomacy?
We have already seen major successes for multi-agents AIs in other adversarial environments such as chess, go and poker 🤖🎮. However, in these games, communication has no value.
In games such as chess, go and poker, communication has no value.
Maintaining human interpretable communication (intentional communication) along with natural language negotiation is challenging. That’s where Diplomacy comes in 🤝
Diplomacy is a strategic board game 🎲 played by seven players who use private natural language negotiations to cooperate and compete with one another. The goal is to coordinate complex plans, but miscommunication can lead to failure as other players may ask the agent to reevaluate their reasoning (potentially causing further errors) or choose to cooperate with others instead.
Game theory is a useful tool for evaluating the effectiveness of advanced techniques in this context.
In games that involve cooperation, self-play without human intervention may not always yield a policy that performs well with humans. This is because self-play often leads to strategies incompatible with human behavior. 🧠
Meta’s Cicero AI successfully demonstrated this by outperforming 90% of human players in the Diplomacy game who had played more than one game and coming in second out of 19 in the league that played five or more games. Overall, across all 40 games, Cicero’s mean score was 25.8%, more than double the average score of 12.4% of its 82 opponents. This achievement demonstrates the power of combining game theory with AI to achieve impressive results. 🏆
CICERO is an artificial intelligence (AI) agent that has achieved human-level performance in the game of Diplomacy, a strategic board game that requires both cooperation and natural language negotiation for tactical coordination. So basically, Cicero combines:
- 💬 A controllable dialogue module: Enables the agent to generate targeted and effective responses in natural language negotiation and tactical coordination.
- 🧠A strategic reasoning engine: Predicts the policies of other players (a probability distribution over actions) for the current turn based on the state of the board and shared dialogue. It then uses this information to choose its own policy for the turn. Doing this with human players requires predicting how humans will play being a popular approach for this, something named behavioral cloning (BC). We’ll see more in further articles.
In other words, at each iteration of the game, this AI models:
- How the other players are likely to act on the basis of the game state and their conversations.
- Then, coordinate and map actions with other players using natural language messages.
It is interesting to remark here that as Diplomacy conversations are private between pairs, Cicero must reason at this stage about what information players have access to when making predictions.
It’s also worth highlighting that CICERO does not predict whether messages from other players are deceptive or not 🙈. Instead, it predicts the policies of other players based on whether a particular strategy would benefit that player and to what extent.
Cicero generates messages through a neural generative Diplomacy dialogue model. We’ll see in the following issues what a neural generative model is but for now, let’s keep the definition from developers.google that says the following:
A generative model includes the distribution of the data itself, and tells you how likely a given example is.
For example, models that predict the next word in a sequence are typically generative models (usually much simpler than GANs) because they can assign a probability to a sequence of words.
Models used:
- 🤖 Imitation dialogue model: It uses R2C2 as a base model (2.7 billion-parameter Transformed-based encoder-decoder model pre-trained on text from the internet using a BART denoising objective. We’ll slice and dice all of this in the following issues.
- 💬 Controllable dialogue model through intents: Standard language modelling approaches would train the dialogue model only to imitate the messages from the dataset but not to outperform them. To go beyond imitation learning, the research team made the dialogue model controllable by generating messages conditioned on a plan specified by the strategic reasoning module. We’ll delve into the details of how this works in future posts, but for now, just know that this is a key part of Cicero’s success in the game of Diplomacy.
According to the research team, the dialogue model is annotated to better understand the goals and motivations behind each message. In order to handle instances where a player’s actions do not accurately reflect the content of their messages (such as when a message is dishonest or plans change), the team implemented a dialogue-conditional action model to predict the most likely action.
And again, Cicero does not explicitly predict whether a message is deceptive or not. Instead, it relies on piKL, an interactive algorithm that predicts player policies based on:
- Conditions on the message and the historical dialog.
- Whether deviating from the BC policy would benefit the player.
This model was trained on a “truthful” subset of the dataset to predict non-deceptive player dialogue. Overall, these techniques have allowed for the creation of a highly effective and versatile AI agent for the game of Diplomacy. 🤖💪
This autumn was a roller-coaster in the tech world due to the latest advances in AI plus, unfortunately, the big layoffs from FANGT companies (let’s include a big T of Twitter 🐦). News worldwide was talking about how the new Twitter CEO laid off half the company’s 7,500 workers 💥 while pushing the remaining employees to quickly build and launch new products. At the same time, Meta also laid off over 11,000 employees in early November. Long story short, the estimate of tech workers that have lost their job is around 200K people already.
If we zoom in into this, a big proportion of this 200K was people whose job was related to AI or data science and here is where I want to go deeper. Starting with the reasons for the layoffs, my thoughts are that there are 2 main causes:
- 💰 Costs vs impact: The wages in Data Science are all times high most of the time in many companies 📈. However, at the same time, it is really hard to measure the impact of data science or AI teams as they aren’t directly related to the income generated. In general, a data scientist will analyze, research and develop technology that will support decision-making in the company (except for companies where their product is an AI itself). So, layoffs in this field won’t impact the survival of the company in the short-term, although it is a clear big impact on the long-term company’s growth 🤔
- 🌪️ Accumulation of world crisis: For years, we have been hit by the effects of a layered crisis. The coronavirus pandemic 🦠 may seem like it didn’t impact the tech industry, but some companies like Zoom saw an increase in market value. However, in the long run, the effects are reaching this field too. Supply chain disruptions ⛓, soaring energy prices ⚡, and rising inflation rates 📈 impact everyone, including tech companies. (Note: I am not an economist, so take my thoughts with a grain of salt).
In conclusion, we’ll see a slight tech deceleration, but the current pace of development is so high that we won’t notice too much. The majority of most successful companies and governments worldwide have clear that investing in AI is a guaranteed ROI 🤑 for them, although maybe maintaining their own AI teams is too expensive, and they might delegate projects to subcontracted companies whose main focus and product was AI services.
What are your thoughts about this? Do you think that we will see an AI winter in the near future? Or do you believe that the interest and investment in AI will continue to grow despite the industry’s challenges? Share your thoughts in the comments below, and let’s start a discussion!
Thank you for reading! I hope you enjoyed learning about ChatGPT’s and the paper review about Cicero. It’s truly a groundbreaking moment in the field of AI and cooperative technology. But what do you think? Do you see any potential applications for this technology in other areas?
I’d love to hear your thoughts in the comments. And don’t forget to stay tuned for the next article in this series.
Thank you again for reading! 👋
Hey there! If you have any questions or want to chat, feel free to reach out to me:
Looking forward to connecting with you! 🤝
