Lifetime Annotations

Introduction

Section Overview: In this section, the speaker introduces the topic of the video and explains how they came up with the idea for it. They also provide some background information about their YouTube channel and what viewers can expect from the video.

Background Information

  • The RUS survey 2019 results showed that people are asking for more learning material about Rust, specifically intermediate level material.
  • Viewers have requested more video content on Rust.
  • The speaker has a YouTube channel where they do live intermediate and uploaded intermediate Rust content.

Purpose of Video

  • The speaker tweeted out to ask what viewers would like to see in a Rust video that is less advanced than their usual ones.
  • Viewers expressed confusion about lifetimes and wanted to see code that uses them in order to understand better.
  • The goal of this stream is to write a bunch of code in Rust covering multiple lifetimes, strings, and generics.

Logistics

  • This stream will be shorter than usual at around 90 minutes.
  • The speaker will take questions from chat throughout the stream.

Getting Started with Cargo

Section Overview: In this section, the speaker explains how to start a new Rust project using Cargo. They also introduce the library they will be building in this video.

Starting a New Project with Cargo

  • To start a new project with Cargo, create a new directory and run cargo init.
  • Inside Cargo.toml, define metadata about your crate and source lib.
  • Add warnings for missing debug implementations, Russ 2018 idioms, and missing docs.

Introducing stir_split Library

  • The library being built in this video is called stir_split.
  • It allows you to take a string and split it by another string.
  • Throughout the rest of the video, we will focus on building out this library.

Introduction to String Split

Section Overview: In this section, the speaker introduces a new type called “String Split” and explains its purpose. They also discuss how to implement the iterator trait for this type.

Building String Split

  • The goal is to create a new type called “String Split” that will have a method called “string split”.
  • The method will take in two parameters: haystack (the string being searched) and delimiter (the string used to split haystack).
  • The method will return an object of a special type that refers to the name of the input lock.
  • Using “self” instead of “that” is useful because it means that if we rename the type later, we don’t have to change all the return types of the methods.
  • The iterator trait allows you to use for loops with your custom types.
  • To implement iterator for String Split, you need to define a next function that takes a mutable reference to self and returns an optional item.
  • A test can be written by creating some string like ABCDE and using stir split on it with space as delimiter.

Lifetimes

Section Overview: This section covers lifetimes in Rust.

Quality Comparison Between Iterators

  • When comparing iterators, Rust checks if all elements are equal.

All Loops Desugar To Loop With A Break Condition

  • While loops desugar into loops as well.

Writing Code for String Split

Section Overview: In this section, the speaker discusses how they plan on writing code for String Split.

Remaining Questions

  • There are still questions about how exactly they plan on implementing certain aspects of String Split.

Implementation of Split Function

Section Overview: In this section, the speaker explains how to implement the split function in Rust.

Implementing Next

  • The implementation of next is straightforward.
  • Find where the next delimiter appears in the remainder and then chop that part of the string.
  • Return everything until the delimiter and set the remainder to what remains after the delimiter.
  • If there is no delimiter in the string, return none.

Using Self as a Preferred Way of Implementing Cascade

  • The speaker prefers using self because it means that if they change the name of the type, they don’t have to change anything else.
  • However, it means you can no longer do local reasoning looking at this line of code.

Associated Types vs Generics

  • Use generics if multiple implementations of that trait might exist for a given type.
  • Use associated types if only one implementation makes sense for any given type.

Match vs If Let’s Sum

  • Use match if you care about more than one pattern.
  • Use if let when you only care about one pattern.

Compiling Errors and Lifetime Specifiers

Section Overview: In this section, we learn about compiling errors and lifetime specifiers while implementing split function in Rust.

Missing Lifetime Specifier Error

  • When trying to compile our code, we get an error message saying “missing lifetime specifier.”
  • We need to give references a lifetime because Rust cannot figure out what that lifetime is on its own.

Adding Lifetime Specifiers

  • We add a lifetime specifier by adding ’a after each reference variable.
  • We use anonymous lifetimes when necessary.

Conclusion and Bug Fixing

Section Overview: In this section, we conclude our implementation of split function in Rust and fix a bug found earlier.

Bug Fixing

  • There is a bug in the code that needs to be fixed.
  • The bug will be addressed later.

Conclusion

  • We have completed our implementation of the split function in Rust.
  • We can now test our code using cargo test.

Understanding Lifetimes in Rust

Section Overview: In this section, the speaker explains how lifetimes work in Rust and why they are important.

How Pointers Work with Iterators

  • Rust calls iterators’ next method and gets back a pointer to a string.
  • Rust needs to know how long it’s okay to keep using this pointer for because it might use that pointer after the string it’s pointing to has already gone away.
  • The tick a is really a lifetime. It represents how long does this reference live for.

Lifetime of References

  • If you have a store split then the remainder and delimiter both live for the same lifetime.
  • The thing that we return has the same lifetime as tick a.
  • Even after you drop the store split, the thing that you get back from the iterator is still valid because it’s about the lifetime of the string we were originally given.

Specifying Lifetimes

  • You can never be wrong by specifying lifetimes. If you specify a lifetime that the compiler thinks is wrong, it won’t let you compile your program.
  • Anonymous lifetimes are places where you tell the compiler to guess what lifetime. This only works when there’s only one possible guess.
  • You can order lifetimes based on how long they are. For example, take static is a ring that lives for the entire duration of the rest of the program.

Conclusion

  • The name you give does not matter; it’s just like naming generics.

Introduction to Lifetimes

Section Overview: In this section, the speaker introduces lifetimes and explains how they work in Rust.

What are Lifetimes?

  • Lifetimes are a way of ensuring that references in Rust are valid.
  • You can use an underscore to tell the compiler not to consider a certain lifetime for guessing purposes.
  • You can specify an order for lifetimes if necessary.
  • The tick underscore is only used if there’s only one possible lifetime.

Anonymous Lifetimes

  • Anonymous lifetimes can be used to simplify code by allowing type inference to determine the correct lifetime.
  • This means that the output position will have its lifetime inferred based on the input position.

Lifetime Errors

  • Lifetime errors occur when there is a mismatch between the expected and actual lifetimes of references.
  • The error message will indicate which reference has an incorrect lifetime and where it was defined.
  • To fix these errors, you need to ensure that all references have compatible lifetimes.

Understanding Lifetimes in Rust

Section Overview: In this section, the speaker discusses the use of generic names for lifetimes and not proper names like typical variables. They also talk about how resilient anonymous lifetimes are and whether relying too much on them can cause trouble.

Generic Names for Lifetimes

  • The speaker explains that generic names are used for lifetimes instead of proper names like typical variables.
  • They mention that anonymous lifetimes are resilient and can be relied upon most of the time.
  • It is possible to give more than one lifetime and then give relationships between them using restrictions.

Relationship Between Lifetimes

  • The speaker mentions that it is possible to give relationships between lifetimes by saying a reference must live longer than another reference.
  • Lifetimes are types, and they can be thought of as subtyping.

Static Lifetime

  • The static lifetime extends until the end of the program, and any value has a lifetime of however long that value is assigned to a variable or moved.
  • If you have a reference or something containing any lifetime, you can assign anything of the same type but with a longer lifetime.

Conclusion

  • The speaker concludes by stating that everything defaults to having a static lifetime, although it’s not entirely true. A value’s lifetime lasts until it’s moved or assigned to a variable.

# Lifetime Annotations

Section Overview: This section discusses lifetime annotations and how they are used in Rust.

Variables and Lifetimes

  • Values only live for as long as they do until it’s moved or dropped.
  • If a value is passed to another function and gets moved, the lifetime of that value ends.
  • The compiler infers the lifetime for every value, but when writing code that is generic over lifetimes, we need to add lifetime annotations to tell it how long we need different pointers to live for.

# String Splitting

Section Overview: This section discusses string splitting in Rust.

String Splitting with Delimiters

  • split() takes a sequence of characters and splits it into multiple smaller strings separated by some delimiter.
  • Using assert_eq! instead of collecting into a vector can provide nicer errors.

Handling Empty Strings

  • Any string written directly in double quotes is compiled into your binary and stored in read-only memory.
  • When there is an empty delimiter at the end of the string, the iterator should produce an empty element.
  • To handle this case, we can use Option and distinguish between whether the remainder is empty or whether the remainder is an empty element we haven’t yielded yet.

# Heap Allocations

Section Overview: This section discusses heap allocations in Rust.

Heap Allocation Lifetime

  • Heap allocations have a lifetime and live until they are dropped.
  • If something is on the heap and never dropped, it would be static.
  • The compiler cannot infer that the type of lifetime we return is tied to the lifetime of the remainder, so we need to add lifetime annotations to tell it how long we need different pointers to live for.

Spotting Allocation in Binary

  • In general, you can spot allocation in binary by dumping the binary.

Fixing a Bug

Section Overview: In this section, the speaker discusses how to fix a bug in the code.

Refactoring Code

  • The speaker suggests using self.remainder.take() instead of return Some(remainder) to simplify the code.
  • The speaker considers using smart matching patterns but decides against it.

Searching for Delimiters

  • If there is some remainder left to be searched, the program will search for the delimiter in that remainder.
  • If the delimiter is found, the program extracts everything before it and sets the remainder to be everything after it. It then returns what was extracted.

Mutable References

  • The speaker explains that ref means “make a new reference or take a reference to” and ref mute means “get a mutable reference to.”
  • The speaker uses ref mute because they want a mutable reference to modify an existing value rather than taking ownership of it.

Understanding References

Section Overview: In this section, the speaker explains references.

Dereferencing

  • The type of remainder here is mutable, but on the right-hand side, it’s not mutable. To assign something from one type to another, we need to dereference it first by adding an asterisk before its name.

Rust Option and Mutable References

Section Overview: In this section, the speaker discusses Rust’s Option type and mutable references. They explain how to use the take function with a mutable reference to an option, and how it can be used to return the remainder of a string that doesn’t have a delimiter.

Using take with Mutable References

  • The take function is implemented on Option<T>.
  • It takes a mutable reference to an option and returns an Option<T>.
  • If the option is None, it returns None.
  • If the option is Some, it sets the option to None and returns the value that was in there.

Simplifying Code with Try Operator

  • The try operator also works on options.
  • We can simplify code using the question mark operator.
  • A let statement is a pattern match.
  • We can use pattern matching on what’s inside the sum of self remainder by taking a reference to what’s in there.

Limitations of Mutable References

  • Mutable references are only one level deep.
  • If you have a mutable reference to self, you’re allowed to modify any of its fields.
  • You cannot change the thing that limiter is pointing to because delimiter itself would have to be a mutable reference.

Multiple Lifetimes in Rust

Section Overview: In this section, the speaker explains multiple lifetimes in Rust. They discuss how lifetime annotations work and provide examples of how they can be used.

Lifetime Annotations

  • Lifetime annotations specify how long references live.
  • They are written as 'a, 'b, etc., where each letter represents a different lifetime.
  • Lifetime parameters are specified inside angle brackets before function arguments or struct definitions.

Writing a Function with Multiple Lifetimes

  • We want to write a function that takes a string and returns the string until the first occurrence of a character.
  • We can use lifetime annotations to specify that the returned string must live at least as long as the input string.
  • We can also use lifetime elision to avoid having to explicitly specify lifetimes in simple cases.

Rust Ownership and Borrowing

Section Overview: In this section, the speaker discusses Rust’s ownership and borrowing system. They explain how it works and provide examples of how it can be used.

Ownership Rules

  • Each value in Rust has an owner.
  • There can only be one owner at a time.
  • When the owner goes out of scope, the value is dropped.

Borrowing Rules

  • You can borrow values by taking references to them.
  • References are immutable by default.
  • You cannot modify a borrowed value until all references to it have gone out of scope.

Mutable References

  • You can also take mutable references to values using &mut.
  • Only one mutable reference is allowed at a time for each value.
  • You cannot have both mutable and immutable references at the same time for the same value.

Rust Smart Pointers

Section Overview: In this section, the speaker discusses smart pointers in Rust. They explain what they are, how they work, and provide examples of how they can be used.

What Are Smart Pointers?

  • Smart pointers are data structures that act like pointers but have additional metadata and capabilities.
  • They implement traits such as Deref and Drop.
  • Examples include Box, Rc, and Arc.

Box Pointer

  • A box pointer is a smart pointer that points to a value on the heap.
  • It is created using the Box::new function.
  • Box pointers are used when you need to allocate memory on the heap and return ownership of it.

Rc Pointer

  • An Rc pointer is a reference-counted smart pointer.
  • It allows multiple owners of the same data.
  • When all owners go out of scope, the data is dropped.

Arc Pointer

  • An Arc pointer is an atomic reference-counted smart pointer.
  • It allows multiple threads to have shared ownership of the same data.
  • When all owners go out of scope, the data is dropped.

Understanding Rust’s Lifetime System

Section Overview: In this section, the speaker discusses Rust’s lifetime system and how it affects code. They explore the issue of returning a reference to a temporary value and how it relates to lifetimes.

Temporary Value Issue

  • Running cargo check shows an error where we’re trying to return a value referencing a temporary value.
  • The error is caused by returning a string reference that is tied to the lifetime of another string.
  • The issue arises because Rust assumes that both strings have the same lifetime, but they don’t.
  • To fix this, we need two lifetimes instead of one.

String vs. Str

  • One option is to store the delimiter as a string instead of a str.
  • Strings are heap-allocated and dynamically expandable while strs are not.
  • A string can be converted into a str easily, but not vice versa.

Conclusion

  • Rust’s lifetime system can be tricky to navigate, especially when dealing with references and temporary values.
  • It’s important to understand the differences between strings and strs in order to write effective code.

String Manipulation

Section Overview: In this section, the speaker discusses string manipulation and the use of heap allocation. They also explain how to fix a problem with multiple lifetimes.

Heap Allocation and String Manipulation

  • The speaker explains that heap allocation is used to copy all characters over to create a string.
  • Storing the delimiter as a string requires an allocation, which is not great for performance. It also ties it to an allocator, making it incompatible with embedded devices.
  • A question is asked about getting a character from until_car and transforming it back into a string. The reference in front of the format produces a string, and ref takes a reference to that.

Multiple Lifetimes

  • The speaker explains that when using references, their lifetime must be tied to the correct object’s lifetime.
  • To fix this issue, multiple lifetimes are needed. Naming these lifetimes allows for generic implementation over haystack and delimiter.
  • By having different lifetimes for haystack and delimiter, the compiler no longer forces them to be the same by downgrading the lifetime.
  • The reference given back is now tied only to the haystack lifetime instead of being tied to both haystack and delimiter.

Conclusion

The speaker discussed heap allocation and its use in creating strings. They also explained how multiple lifetimes can be used when dealing with references.

Introduction to Lifetimes and Delimiters

Section Overview: In this section, the speaker discusses how a reference with a lifetime delimiter can be downgraded to the lifetime of another variable. They also introduce the concept of delimiters and how they can be used to split strings.

Lifetime Delimiter Downgrading

  • A reference with a lifetime delimiter can be downgraded to the lifetime of another variable.
  • The reverse is not true.

Introducing Delimiters

  • Delimiters are introduced as a way to split strings.
  • The delimiter can be any type that can find itself in a string.

Implementing Delimiter Trait

  • A trait called “limiter” is introduced for delimiters.
  • The limiter trait requires that the delimiter must be able to give us its length and skip past it.
  • We implement iterator first or split for any D where D implements delimiter using find next method.
  • We implement delimiter for a reference to a string s by finding the start and end indices of the substring within s using s.find(self).

Generic Delimiter Implementation

Section Overview: In this section, the speaker demonstrates how to implement a generic delimiter for any type D that implements the limiter trait. The implementation is done in such a way that it allows for references to be passed without requiring a lifetime.

Implementing Generic Delimiter

  • Implemented generic delimiter over any type D that implements the limiter trait.
  • This implementation allows for references to be passed without requiring a lifetime.
  • Demonstrated how to implement delimiter for character using s.car_indices() method.
  • Iterated over all characters of the string looking for one that is the character we’re searching for and then when it finds whatever results it finds if it finds one we map that sum to take the position and return that position and that position plus one right because the character is only one character long.
  • Implemented this pattern for all sorts of other types so anything that can find itself in a string will now just work.

Q&A

Section Overview: In this section, the speaker answers some questions from viewers about various aspects of the implementation.

Questions and Answers

  • Explained what self refers to in pattern self here in the implementation down here. It’s a reference to this type so it’s a reference to a reference to Astor.
  • Discussed whether there is a simpler way than character indices. There is but this shows the concept of it like you can do way more efficient things than this.
  • Clarified whether find(self) returns self or its index. Find as a method on strings that you can give it a string and it will tell you the start of that string in that string that plus one is wrong and will panic your code.
  • Explained why self.len() and not s.len(). Self is the thing we’re searching for, it’s the length of the delimiter. In order to find the end of the delimiter, it has to be the start of the delimiter plus the length of the delimiter not the length of the string we’re searching in. The s here is what we’re searching in.

Conclusion

Section Overview: In this section, the speaker reveals that all of the things implemented today exist in Rust’s standard library.

  • All of the things implemented today exist in Rust’s standard library.
  • Split on a string takes a reference to self and some pattern P and it returns to split. If we look at split split implements iterator and it gives you things split by that.
  • Today’s implementation goes through where multiple lifetimes are useful and how to turn these kinds of implementations into something more generic.

Rust Standard Library Exercise

Section Overview: In this section, the speaker discusses the exercise of implementing patterns and understanding how different pieces fit together in Rust.

Implementing Patterns

  • The exercise is not meant to be published as a crate because it’s already in the standard library.
  • The exercise helps understand how different types of lifetimes work and when multiple lifetimes are needed.
  • Lifetime errors and differences between strings and stores are also covered.

Questions and Answers

Section Overview: In this section, the speaker answers questions from viewers related to Rust programming language.

Creating Strings from Astor Fat Pointer

  • You cannot create a string from Astor fat pointer because you don’t own the memory. A string assumes that it owns the underlying memory, which is not true for arbitrary pointers.

Readability of Rust Programming Language

  • The speaker does not think that Rust is less readable than other languages. Additional features in Rust require additional syntax, but they add more functionality.

Pattern in Haystack

  • The pattern in haystack shares the same lifetime tick a reference to the haystack. It is similar to what was done during the exercise.

Generic Associated Types (GAT)

  • GAT will help with being able to clone less. It may not necessarily help with trait definitions, but it will help with existential types and other things.

Future Plans for Videos on Rust Programming Language

Section Overview: In this section, the speaker talks about future plans for videos on Rust programming language.

Beginner Streams

  • The speaker is not planning to do any complete beginner streams, but might do more focused videos if there is an appetite for them.

Conclusion and Farewell

Section Overview: In this section, the speaker concludes the presentation and thanks the audience for attending.

  • The speaker thanks everyone for joining and hopes that they learned something from the presentation.
  • The presentation being 90 minutes instead of six hours made it more digestible.
  • The speaker encourages everyone to stay safe and stay home.
  • The speaker announces that there will be a video next time.

Generated by Video Highlight

https://videohighlight.com/video/summary/rAl-9HwD858