I’m currently working on a codebase using two crates exposing a very similar enum 

, representing some sort of priority level, say “LOW”, “MEDIUM”, “HIGH” and “CRITICAL”. 

, but the orphan rule prohibits me from implementing 

use external_crate::Priority;

// This is what we refer to as the "newtype"
struct PriorityWrapper(external_crate::Priority);

// We can now implement the foreign trait Into for our _local_ type PriorityWrapper
impl Into<u8> for PriorityWrapper {
    fn into(self) -> u8 {
        match self.0 {
            Priority::LOW => 1,
            Priority::MEDIUM => 2,
            Priority::HIGH => 3,
            Priority::CRITICAL => 4,
        }
    }
}

// Example usage
fn print_a_priority(priority: PriorityWrapper) {
    let number: u8 = priority.into();
    println!("Priority as number: {}", number);
}

Although I got around the orphan rule using this pattern, I was not entirely satisfied with the solution.

As an alternative to using the newtype pattern, I've experimented with creating a new trait instead: 

. Then I could implement this trait for the external enums. This way, I still use the underlying type, but I also have a new method 

use external_crate::Priority;

// This is not a "newtype", but a new trait instead!
trait PriorityExt {
    fn convert_to_u8(&self) -> u8;
}

// It is quite similar to the Into<T> trait
impl PriorityExt for external_crate::Priority {
    fn convert_to_u8(&self) -> u8 {
        match self {
            Priority::LOW => 1,
            Priority::MEDIUM => 2,
            Priority::HIGH => 3,
            Priority::CRITICAL => 4,
        }
    }
}

// Example usage
fn print_a_priority(priority: Priority) {
    let number: u8 = priority.convert_to_u8();
    println!("Priority as number: {}", number);
}

Now do we call this “newtrait pattern” or what?

I just jumped into the Rust Discord and got informed that this technique is more commonly known as “extension traits”. Although not mentioned in the Rust Book, it seems this pattern has been gaining popularity and being mentioned in the 

Comparing extension traits to the newtype pattern

Why would you use extension traits instead of the newtype pattern to work around the orphan rule?

One great benefit from using extension traits is the improved developer experience compared to newtypes

You would not be mistaking the newtype from the underlying type

No need to do difficult naming of the wrapping newtype

No need to re-expose the methods on the underlying type

One disadvantage of extension traits is that you have to import the trait to be able to use it. The language server suggests this for you however, with automatic code patching as well, making it a non-issue for those using a modern IDE.

Well, I was searching for a solution to the orphan rule, and nowhere was extension traits mentioned as a solution. A quick search for “rust orphan rule traits” yields no results mentioning extension traits (at the time of writing).

The Rust Book only highlights the newtype pattern as a solution to the orphan rule in the chapter about 

. And Karol’s blog post compares the language feature to C#.

But extension traits for me has been an elegant workaround for the orphan rule!

Well, now it’s out there! 💁‍♂️

Why use a trait at all? This could be a function

For sure! Using a function for a simple type conversion would be perfectly fine. But if you’re working with multiple “similar” foreign types, a trait could help align those behind a common API.

This would also not give you any code completion on the foreign type when hitting “.”, making the solution a bit more hidden for future developers.

Newtype or newtrait? Ways around the orphan rule

In this article, I'll explain to you how I used OpenAI embeddings and completions API to implement an AI chatbot answering questions for employees at our company. The code examples I'll show are written in Typescript, but the same principles apply to any programming language.

Short intro about the problem we're trying to solve here

I work in a IT consulting company called 

. We're around 60 employees, and have a quite large internal document we call 

, which has a lot of useful information about employee benefits, salary, how to book travel, how to use our internal systems, and so on. As a company in growth, this document grows accordingly. Finding the correct information can get increasingly difficult.

Maybe a chatbot can make this information more available for our employees?

I guess the rudimentary solution would be to simply paste the contents of our handbook into a chat with 

Given the following internal document in our company:

"""
<PASTE_HANDBOOK_HERE>
"""

Answer the following question from an employee: "How do I book travel?"

This however did not produce satisfactory results. If we ask a question about something in the first chapters of the handbook, the AI simply did not answer correctly.

The AI can only read a limited amount of text before answering. This is more precicely called the "context size" of the language model.

The context size for GPT-3 is 4092 tokens, which is a lot less than the handbook, which is well above 30 000 tokens. The newer GPT-4 model has a context size of 32 768 tokens, which would be enough today, but not in the future.

It seems like we have to limit the amount of text the AI has to read before answering. To put it simply, we need to implement some kind of search, and provide the search results to the AI.

Replacing keyword-based search with AI-enabled 

If you've implemented a search engine before, you know that there are many ways to do this. The most common way is to implement a keywords-based search where you simply look for the exact words from the query in the database. If you want to get fancy with it, you could also implement some sort of weighting of the words. For instance, you could make prepositions less important than names. From there, you could continue making the search engine more and more advanced, but it will still be based on keywords-matching.

With the advent of AI, we can now implement a search engine that is not based on keywords, but on 

. This means that we can search for the 

 of the query, instead of the exact words. This is called 

To be precise, the Large Language Model (LLM) can read some text and produce a vector representation of the meaning of the text. This vector is called an 

We can then use this embedding to compare the meaning of the query with the meaning of the text in our database. The result is a list of documents sorted by how similar they are to the query.

Here is a high level overview of what we're going to build.

An employee writes a query to the AI, and we send it to OpenAI to receive an embedding vector of it

We then query our vector database (for example 

) to find the most similar pieces of text to this query. This is the 

We then send the search results to OpenAI completion API to produce an answer

This implies that we've split our internal documentation into smaller chunks, and created embeddings for these in our database. Creating an embedding is somewhat expensive, so I recommend creating a script for the initial creation of the embeddings, and then update the embeddings when the chunks changes.

Splitting the contents into smaller chunks

In our case, the handbook content was stored in Notion. I'm not going to go into detail on how to fetch the content from Notion, but I'll assume you know how to fetch content from your CMS.

At this point you'll have a large piece of text, or several large ones. However, we need to split it into smaller chunks. In our case, we simply split the document into paragraphs. The result is a list of chunks that looks something like this:

[
    "We have a lot of benefits for our employees. Here are some of them...",
    "If you are on sick leave, you will get 100% of your salary...",
    ...
]

This particular step is kind of important to get right. In my case, a lot of effort went into this. However, it's also very specific to our case and Notion APIs, and thus outside the scope of this article.

We'll be using OpenAI to create embeddings. You could interface with their REST API directly using 

, however, for simplicity sake we'll use the 

Then let's write a function that takes in the text content and returns an embedding vector:

import OpenAI from 'openai';

const OPENAI_API_KEY = process.env.OPENAI_API_KEY
const EMBEDDING_MODEL = "text-embedding-ada-002"

async function createEmbedding(content: string): Promise<number[]> {
  const openai = new OpenAi({
		apiKey: OPENAI_API_KEY,
	});

  const response = await openai.embeddings.create({
    model: EMBEDDING_MODEL,
    input: content,
  });

  return response.data[0].embedding
}

In the code example above, we chose to use the embedding model called 

. You could replace this with any other embedding model from OpenAI.

If you wonder what embedding model to choose, I'll recommend just testing them out and see which one performs best for your use case. Start with the cheapest one and move to a heavier one if it's not good enough 💸

There are many options for storing the embeddings. In our case, we chose to use 

. You can get a QDrant server up and running very quickly using Docker:

docker run --rm -p 6333:6333 -v $(pwd)/qdrant_storage:/qdrant/storage qdrant/qdrant

Now you have the QDrant API listening on port 

 locally. Interfacing with QDrant could be done directly using 

export function insertEmbeddings({
  collectionName,
  embeddings,
}: {
  collectionName: string;
  embeddings: Embedding[];
}) {
  return fetch(`${QDRANT_URL}/collections/${collectionName}/points`, {
    method: 'PUT',
    headers: {
      'Content-Type': 'application/json',
      'api-key': QDRANT_API_KEY,
    },
    body: JSON.stringify({
      points: embeddings.map((e) => ({
        id: randomUUID(),
        vector: e.embedding,
        payload: {
          // This is where you can store metadata about the embedding
        },
      })),
    }),
  });
}

Vectors are stored in separate collections, so you have to create a collection using their 

At this point you have chunked your content into smaller pieces, created embeddings for each of them, and stored them in Qdrant. Then it's time to perform the semantic search. Let's reuse our 

 function as defined in our snippet above, and use 

async function search({
  query,
  collectionName,
}: {
  query: string;
  collectionName: string;
}) {
  // First we create an embedding for the query
  const vector = await createEmbedding(query)

  // Then we find the top 5 most semantically similar chunks to our query
  const response = await fetch(`${QDRANT_URL}/collections/${collectionName}/points/search`, {
    method: 'POST',
    headers: {
      'Content-Type': 'application/json',
      'api-key': QDRANT_API_KEY,
    },
    body: JSON.stringify({
      vector,
      limit: 5,
      with_payload: true, // This will return the metadata we stored in the payload
    })
  });

  const searchResponseJson = await searchResponse.json();

  return searchResponseJson.result
}

Using this function you'll get a list of chunks that most likely contain an answer for the query. Now it's time to send these chunks to OpenAI to get an answer.

async function getAnswer({
  query,
  collectionName,
  openai,
}: {
  query: string;
  collectionName: string;
  openai: OpenAI
}): Promise<string> {
  // First we perform the semantic search
  const searchResults = await search({ query, collectionName })

  /**
   * Then we put the search results together with the question 
   * from the employee and send it to OpenAI for answering
   */
  const response = await openai.chat.completions.create({
      model: 'gpt-3.5-turbo',
      messages: [
        {
          role: 'system',
          content:
            'You are an assistant who answers questions for employees in a consultancy firm called Fink. You do not ask any questions.',
        },
        {
          role: 'user',
          content: `${searchResults.map(
            (r) =>
              `"""Here are some texts from our handbook:\n${
                r.input_text
              }\n"""\n`
          )}\n\nQuestion: ${question}`,
        }
      ],
      temperature: 0.1,
    })

  return response.choices[0].message?.content
}

And that's it! There is a lot of glue code left out in this article for brevity, but in short this is what we've done:

Put the search results together with the question and send it to OpenAI for answering

And that's it! We made a simple UI for our AI and this is the result:

I made an AI chatbot answering questions for employees at our company

This fall, I've been assisting the comedy group 

 in setting up a show where improvisers and the audience interact with AI live on stage.

We have shows every third Tuesday of the month, and the next one is on 

In case you're not quite sure what improv theater is, it simply involves creating a comedy show without a script.

The improvisers receive suggestions from the audience and create a story on the spot. It's a very fun and interactive form of theater, and the audience gets to contribute to what happens on stage.

What's different about the concept of Kunstig Humor is that AI also contributes to what happens on stage. Specifically, large language models like GPT-4 and image generation with DALL·E are used as inputs for the improvisers.

The show culminates in a competition between AI and humans to be the funniest. The audience selects the winner 🏆

I'm not an improviser myself, but I contribute as the character "CryptoJack". I'm a hacker who has managed to hack into the AI systems and can thus provide input to the improvisers 👨🏼‍💻

At least that's the official story. To put it more precisely, in technical terms, I've developed some fancy hacker-looking interfaces on top of OpenAI's APIs, quite similiar to what we're all familiar with on 

Four shows down the road, and I've learned a lot about "live coding" on stage. We all know the demo gods are not always on our side, and I've had my fair share of bugs and issues.

First of all, you can't control what the AI is going to produce, and that kinda keeps me on the edge through the show, haha. I guess that just fits neatly into the improv concept, but it's still a bit thrilling when you're on stage and the AI produces something that way different than what you got when prompt engineering the day before.

There's always some technical preperations to be made, so it's always smart to add at least 20 minutes of buffer time before the show starts. I've had to replace HDMI cables, fight with a decades old projector, and fix some last-minute bugs.

Last but not least, every mistake can be funny! I've had some bugs that have made the audience laugh, and I've learned to embrace it. The improvisers also certainly knows how to exploit them! It's a comedy show after all.

Kunstig Humor – Improv theater meets AI

I wanted to learn about 3D game development, so I set out on a small project inspired by a friend of mine who lives on a farm. His name is Gunnar and he lives on a farm called 

Growing up we spent a lot of time up on his farm, doing all kinds of activities. Most recently we started brewing beer and have a lot of fun experimenting with different recipes.

This article however, is about my tiny project, recreating his farm and its atmosphere in a game. I had to leave it in an unfinished state, as I quickly realized I had not sucessfully scoped it down to a realistic workload for a solo developer 😅

Set against the backdrop of a calm farm, the game offers a peaceful gameplay experience. The farm's visuals might be on the low-poly side, but I've tried to maintain a consistent and familiar feel throughout.

On this farm, players can talk with three NPCs. One of them, a fishing enthusiast, invites you to a brief mini-game. The objective is simple: catch as many fish as you can in two minutes using a straightforward spacebar mechanism. It's a modest challenge.

I used Godot as game engine, which shares similarities with platforms like Unity. I used Blender for 3D modeling and this was quite challenging.

While "A Happy Farm" might not see further updates, the insights gained will sure influence my next game development project.

My first stab at 3D game development

If you are a newcomer to Rust, you might be interested in how you should properly organize your project files and folders.

 I collected data from GitHub REST API to look for large patterns in how most Rust projects organize their files and folders. The results however, were kinda expected. So that's why I decided in this article to go for a more qualitative approach.

Instead of going through hundreds of projects, I'm going to manually look for patterns in a smaller selection of projects.

I wanted this article to include practical advice based on observations in the projects I went through.

 allows you to separate your code into smaller crates, instead of having them as modules in a single crate.

If you want to use this feature, I suggest you create a top level 

 folder and put your workspace members inside here. This way, you could include the following entry in Cargo.toml:

# Cargo.toml

# ...other stuff hidden for brevity

[workspace]
members = ["crates/*"]

That said, there is nothing wrong about organizing this differently.

Let's look at three of the most popular game engines written in Rust. They have each decided to put the workspace members different places:

 have these in its project root folder, all prefixed with 

Piston chose to put this into subfolders inside 

I don't see any practical differences in how you organize your workspace members into folders. I would however appreciate some sort of suggested default, so this becomes more recognizable at a glance, instead of having to open 

 or something that automatically patched 

 with the new member crate inside it. Like this:

$ tree
.
├── Cargo.toml
├── src
│   └── main.rs
└── crates
    ├── project-core
    │   ├── Cargo.toml
    │   └── src
    │       └── lib.rs
    ├── project-log
    │   ├── Cargo.toml
    │   └── src
    │       └── lib.rs
    └── project-ui
        ├── Cargo.toml
        └── src
            └── lib.rs

8 directories, 9 files

Where should I put examples on how to use my project?

I would suggest you create a root level 

# Cargo.toml

# ...other stuff hidden for brevity

[[example]]
name = "hello_world"
path = "examples/hello_world.rs"

If you do not have any examples, I would encourage you to create some. They're super helpful for learning about you project quickly.

Eight of the projects I went through had a top level 

 folder. Some projects (e.g. Macroquad) have this in a separate repo and links to this from the README.md instead.

Having examples is obviously more common for library type of projects, and less for binaries like games, command line utils, etc.

Where should I put my custom error types?

 file which defines a custom enum. Here's an example of such enum, copied from the source code of 

// server/src/error.rs
pub enum Error {
    NetworkErr(NetworkError),
    ParticipantErr(ParticipantError),
    StreamErr(StreamError),
    DatabaseErr(rusqlite::Error),
    PersistenceErr(PersistenceError),
    Other(String),
}

 I found that 23% of Rust projects on GitHub have a 

 folder. This time, when I manually inspected the projects I found this file in more than half of the projects. It's not always inside the 

 folder because of cargo workspaces and custom folder structures. Some projects put them inside a subfolder 

Bevy had this unique way of putting error codes in the form of 

) have its own markdown file explaining the error.

When looking through the projects, I also observed some nice tools and techniques that I figured would be nice to share.

 is done before commits are pushed. It will prevent your git history from containing commits like "Fix formatting", "Fix broken unit test" etc. because someone forgot to run these commands before merging.

 to prevent unwanted dependencies to creep into your project. You may want to do this to keep known vulnerabilities or unwanted licensing out of your project. This will involve creating a file 

 in your project, containing this blacklist of crates. 

 to ensure a coherent format across you project. This is kind of the same as using a 

 file, but it has more features as it's configuring 

, you'll ensure a clean, coherent code style across all source code.

I included the few open source Rust games on 

I found a great list of command line utilities written in Rust on another tech blog 

I'll recommend people to manually look at the projects themselves. It is quite difficult to convey the insight I gained in an entertaining manner.

How most Rust projects are organized (Part 2)

As I'm learning Rust, I regularly ask myself how a more seasoned Rust developer would solve problem X. It's not a matter of finding a solution, but choosing the most popular one, so that my code becomes more recognisable for other Rust developers.

How should I organise the files and folders in my project?

In order to find out how most people structure their Rust project, I decided to look at the masses. I wrote a simple python script collecting ~1000 Rust repos from GitHub, and counted files by name looking for patterns. If you want more info on how exactly I did it, jump to the bottom of this article.

I ran the script looking for common files and subfolders inside 

It's worth noting that 335 repos were skipped because they did not have a 

folder. These probably have a subfolder with a 

$ python3 analyze_repo_structure.py --subfolder /src --percentage-treshold 2
335 repos were skipped because they did not have a `src` folder.

Most common files and folders inside src in a selection of 665 projects written in `rust`:
(67%) 📄 lib.rs
(41%) 📄 main.rs
(19%) 📄 error.rs
(15%) 📄 config.rs
(10%) 📁 bin
(9%)  📄 macros.rs
(9%)  📄 util.rs
(8%)  📄 utils.rs
(5%)  📄 cli.rs
(4%)  📄 app.rs
(4%)  📄 errors.rs
(4%)  📄 prelude.rs
(4%)  📁 utils
(4%)  📄 input.rs
(3%)  📄 types.rs
(3%)  📁 util
(3%)  📄 client.rs
(3%)  📁 config
(3%)  📁 ui
(3%)  📄 context.rs
(3%)  📄 parser.rs
(3%)  📄 server.rs
(2%)  📄 args.rs
(2%)  📄 options.rs
(2%)  📄 event.rs
(2%)  📁 core
(2%)  📁 tests
(2%)  📄 buffer.rs
(2%)  📄 logger.rs
(2%)  📄 style.rs
(2%)  📄 version.rs
(2%)  📁 test
(2%)  📄 result.rs
(2%)  📄 color.rs
(2%)  📄 node.rs
(2%)  📄 request.rs
(2%)  📄 response.rs

Well that's a long list of generic, but somehow recognizable files and folders. The top two is somewhat expected. However, 

, it covers 23% of all rust repos I went through. This is interesting. It seems like a lot of projects put their custom 

 crate is also doing this. I think I'll begin following this pattern. Especially for library crates.

 made it on the list. I'm often creating this file myself for defining my program configuration struct.

 folder is for repos that have more binaries than the default 

Following this, we have a list of kinda generic words that I can imagine people use for different purposes. I wonder what 

There might be interesting stuff outside 

We'll probably see a lot of common files like 

$ python3 analyze_repo_structure.py --percentage-treshold 2
Most common files and folders in a selection of 1000 projects written in `rust`:
(98%) 📄 .gitignore
(96%) 📄 README.md
(93%) 📄 Cargo.toml
(80%) 📁 .github
(66%) 📁 src
(56%) 📄 LICENSE
(49%) 📄 Cargo.lock
(41%) 📄 CHANGELOG.md
(39%) 📁 tests
(35%) 📁 examples
(33%) 📄 CONTRIBUTING.md
(27%) 📄 rustfmt.toml
(26%) 📄 LICENSE-MIT
(24%) 📄 LICENSE-APACHE
(23%) 📁 docs
(20%) 📄 build.rs
(18%) 📄 .travis.yml
(18%) 📄 CODE_OF_CONDUCT.md
(16%) 📄 .gitattributes
(16%) 📁 .cargo
(15%) 📄 Makefile
(14%) 📁 benches
(14%) 📁 scripts
(14%) 📄 .editorconfig
(12%) 📄 .dockerignore
(11%) 📄 .gitmodules
(10%) 📁 ci
(10%) 📄 Dockerfile
(9%)  📄 .rustfmt.toml
(9%)  📁 assets
(8%)  📄 rust-toolchain
(7%)  📄 clippy.toml
(7%)  📁 .vscode
(7%)  📄 SECURITY.md
(6%)  📁 doc
(6%)  📁 crates
(6%)  📁 tools
(5%)  📄 LICENSE.md
(5%)  📁 fuzz
(4%)  📄 deny.toml
(4%)  📁 docker
(4%)  📄 appveyor.yml
(3%)  📄 bors.toml
(3%)  📄 codecov.yml
(3%)  📄 package.json
(3%)  📄 LICENSE.txt
(3%)  📄 rust-toolchain.toml
(3%)  📁 core
(3%)  📁 test
(3%)  📁 .circleci
(3%)  📁 bin
(3%)  📁 resources
(3%)  📄 .clippy.toml
(3%)  📄 Cross.toml
(2%)  📁 cli
(2%)  📁 contrib
(2%)  📁 images
(2%)  📁 lib
(2%)  📄 release.toml
(2%)  📁 book
(2%)  📁 config
(2%)  📄 ARCHITECTURE.md
(2%)  📄 Makefile.toml
(2%)  📄 RELEASES.md
(2%)  📁 data
(2%)  📁 snap
(2%)  📄 .gitlab-ci.yml
(2%)  📄 .codecov.yml
(2%)  📄 COPYRIGHT
(2%)  📄 shell.nix
(2%)  📄 COPYING

Yet again, no surprises in the top of the list.

 folder mostly contain integration tests as unit tests tends to be written alongside the source code in most of the Rust projects I've looked at.

 are nice, especially for library crates.

Fun fact on continuous integration: it seems like 80% are using GitHub Actions and 19% are using Travis CI.

First, we need to collect a list of Rust projects from GitHub. I did this using the GitHub REST API 

 on this API is 100, so we need to use the pagination to collect more than 100 repos.

def get_repo_list(args):
    repos = []

    page = 0
    per_page = 100
    if args.repo_count < 100:
        per_page = args.repo_count
    
    while len(repos) < args.repo_count:
        response = requests.get(
            'https://api.github.com/search/repositories',
            params = {
                'q': 'language:rust',
                'per_page': per_page,
                'page': page
            }
            )
        response.raise_for_status()
        result = response.json()
        
        repos += result["items"]
        page += 1
    
    return repos

Now we have a large list of repos, so let's start counting file and folder entries on them. We do this using the 

for repo in repos:
    response = requests.get(
        '%s/contents/%s' % (repo["url"], args.subfolder),
        headers = headers
        )
    if response.status_code == 404:
        skipped_repos += 1
        continue
    
    response.raise_for_status()
    contents = response.json()

    total_repos += 1
    for item in contents:
        name = ""
        if item["type"] == "file":
            name += "📄 " # Oh yes, I'm using emoji in source code 😎
        else:
            name += "📁 " # Oh yes, I'm using emoji in source code 😎
        name += "%s" % (item["name"])

        if file_count.get(name):
            file_count[name] += 1
        else:
            file_count[name] = 1

Finally, I wrapped these loops with proper argument parsing, caching to spare Github of excessive traffic and some error handling.

 If you want to run this script yourself, I recommend setting up a 

 and provide it to the script with the 

 option. Without authorization, you will only be able to fetch 60 repos, and that would not be a large data set.

How most Rust projects are organized

If you've been in the woods communicating with others using radios, you'll be quite familiar with the term 

If someone is talking on the radio, it broadcasts to everyone. The next person can speak only when the first person is done.

So if someone puts their radio in their not-so-spacious backpack, the "press to talk"-button might be pushed down permanently. And there you have it. Backpack noises broadcasting live. The entire radio network is rendered useless.

Quite fragile? Well, there are many solutions to this, and the problem can be easily eliminated.

However, the consept of "one person broadcasts at a time" is still the standard for radios. This is useful in some cases, but also has some shortcomings

 – If you're out of reach when something is said, you'll miss it. Period. You cannot replay. The information is lost into the ether, as radio waves gets absorbed by the trees and atmosphere.

 – If there's an ongoing conversation, and you have some information that is not urgent, you probably don't want to disturb the ongoing conversation. As there is no other option but to wait, you go to make some coffee. By the end of it, you forgot what it was all about.

 – Your radio beeps with a "small battery"-warning. You decide to change batteries. Waiting for the radio to reboot you start to wonder: "Did I miss anything while my radio was offline?"

These shortcoming becomes more noticable as the group grows and more radios are added to the network.

I was tinkering with my radio and found a RJ-45 interface. The radio I was using was a Kongsberg MH600 VHF radio. It's a bit heavy, but quite reliable. Luckily, it supports sending and receiving IP traffic.

So I went to the drawing board and roughly sketched out something that kind of looked like Twitter. A message input, a message queue and a message feed.

The design is simple, yet effective for solving the issues described above:

 – The message queue with automatic retries will ensure that your messages are received by others.

 – You no longer have to wait for conversations to finish before sending message.

 – The feed will give you a better overview of the ongoing conversation.

Now let's dive into the technical details. The first thing to ask ourselves is: how do we even send a message to the other radios? We'll need to have some sort of reliable data transfer (RDT), so let's try out a simple TCP socket.

Hmm, looks like we're having a lot of spurious retransmissions. It's taking up a lot of our bandwith actually. We have to get rid of this.

Looking further into it, it seems like the TCP implementation of Windows (as we we're testing on here) simply cannot handle the extreme volatility in round trip time (RTT) between the radios. At one point, RTT was 100ms, the next second, it was 1300ms. The 

 in TCP does not tolerate this variance in RTT, resulting in retransmission of packets that are in transmission.

Okey, so we either ditch TCP or reimplement TCP in user space.

So then we're opting for UDP instead. This requires us to implement some mechanism to ensure RDT, as UDP is simply "fire and forget". Packet loss is simply not handled by UDP, and packet loss 

How could we implement reliable data transfer (RDT)?

In my case, there is always at least one radio that has a direct connection to all other radios. In the sketch below, it is radio A.

This makes everything easier, as one radio has to be the authrotitative source of truth. This way, deciding "what has been sent or not" can be left to just asking this radio. If radio A has not received your message, it is not considered "received".

If your network does not have a radio with connection to everyone, I would advise you to establish a repeater or something like that. It just makes everything a lot easier if you have an authoritative source of truth. Or let's just call this "the server".

If all radios have a direct connection to every other radio, then we'll just pick one. Preferrably the one in the center geographically.

How to reliably send messages to the server radio?

As we now have one radio to act as a server, we'll implement the RDT for sending messages to this server. The rudimentary implementation of this is simple: client keeps sending message until a receival receipt is returned from the server, more commonly known as ACK-messages.

I enjoy making games sometimes

I also know a thing or two about radios and wireless communication

Nowadays I work with passenger information on public transportation