Add references to Sway (#5406)

## Description

This PR brings
[references](https://github.com/FuelLabs/sway-rfcs/blob/ironcev/amend-references/files/0010-references.sw)
to the language. The overall effort is tracked in #5063.

In the below description, when talking about references we mean `&T`
references - references to immutable values. `&mut T` will be
implemented in upcoming PRs.

The PR implements the following features:
- References exist in the type system and can be declared, aliased,
dereferenced using `*` operator, etc.
- Reference expressions (referencing) are fully supported on all
expressions including the nonsensical one like e.g. `&return;`
- Semantic analysis and checks are implemented for the current
functionality.
- References can be embedded in aggregates.
- References can reference parts of aggreagates.
- References can be passed to and returned from ASM blocks.
- References can be passed to and returned from functions.
- References can be mutable: `let mut r_x = &x`
- Impls can be implemented for non-generic reference types.
- Traits can be implemented for non-generic reference types.
- References work with generics, means `&T` works as expected.
- References can reference references.
- References can be dereferenced using the `*` operator.
- Dereference expressions (dereferencing) are fully supported on all
expressions including the nonsensical one like e.g. `*&return;`

Known limitations:
- When declaring references on references, currently a space (' ') is
mandatory between two ampersands. E.g., `&&a` emits error and must be
written as `& &a`. Extending parser to support, e.g., arbitrary
`&&...&&a` will be done in upcoming PRs.
- Referencing function parameters for copy types works but not with 100%
proper semantics (creates a copy of the parameter).

On the IR level, references are pointers stored as `u64` and do not
exist as a separate concept. They play well with all the existing IR
optimizations.

Since `core::ops::Eq` and `__addr_of()` are still not implemented for
references, checking for equality in tests is done by converting
references to `raw_ptr`s and checking raw pointers. This is essentially
fine and will one day be turned into converting references to typed
pointers which are tests we will anyhow want to have.

Next steps are listed in #5063.

The documentation will be done in a separate PR and provided separately
once feature is considered ready for public usage.

There are several todos in code marked as `TODO-IG`. They will be
resolved in upcoming PRs.

# Demo

For extensive demos of implemented features, see the tests in this PR.

An example of a semantic check:

![Expression cannot be
dereferenced](https://github.com/FuelLabs/sway/assets/4142833/0d1a3576-3725-4a70-abf2-e5e11625f429)

## Checklist

- [x] I have linked to any relevant issues.
- [x] I have commented my code, particularly in hard-to-understand
areas.
- [ ] I have updated the documentation where relevant (API docs, the
reference, and the Sway book).
- [x] I have added tests that prove my fix is effective or that my
feature works.
- [ ] I have added (or requested a maintainer to add) the necessary
`Breaking*` or `New Feature` labels where relevant.
- [x] I have done my best to ensure that my PR adheres to [the Fuel Labs
Code Review
Standards](https://github.com/FuelLabs/rfcs/blob/master/text/code-standards/external-contributors.md).
- [x] I have requested a review from the relevant team or maintainers.

sway-ast/src/expr/mod.rs

@@ -68,11 +68,11 @@ pub enum Expr {
         field_span: Span,
     },
     Ref {
-        ref_token: RefToken,
+        ampersand_token: AmpersandToken,
         expr: Box<Expr>,
     },
     Deref {
-        deref_token: DerefToken,
+        star_token: StarToken,
         expr: Box<Expr>,
     },
     Not {
@@ -227,8 +227,11 @@ impl Spanned for Expr {
             Expr::TupleFieldProjection {
                 target, field_span, ..
             } => Span::join(target.span(), field_span.clone()),
-            Expr::Ref { ref_token, expr } => Span::join(ref_token.span(), expr.span()),
-            Expr::Deref { deref_token, expr } => Span::join(deref_token.span(), expr.span()),
+            Expr::Ref {
+                ampersand_token,
+                expr,
+            } => Span::join(ampersand_token.span(), expr.span()),
+            Expr::Deref { star_token, expr } => Span::join(star_token.span(), expr.span()),
             Expr::Not { bang_token, expr } => Span::join(bang_token.span(), expr.span()),
             Expr::Pow { lhs, rhs, .. } => Span::join(lhs.span(), rhs.span()),
             Expr::Mul { lhs, rhs, .. } => Span::join(lhs.span(), rhs.span()),

sway-ast/src/keywords.rs

@@ -75,7 +75,6 @@ define_keyword!(LetToken, "let");
 define_keyword!(WhileToken, "while");
 define_keyword!(WhereToken, "where");
 define_keyword!(RefToken, "ref");
-define_keyword!(DerefToken, "deref");
 define_keyword!(TrueToken, "true");
 define_keyword!(FalseToken, "false");
 define_keyword!(BreakToken, "break");
@@ -85,12 +84,12 @@ define_keyword!(TypeToken, "type");
 define_keyword!(PtrToken, "__ptr");
 define_keyword!(SliceToken, "__slice");
 
-/// The type is a keyword.
+/// The type is a token.
 pub trait Token: Spanned + Sized {
-    /// Creates the keyword from the given `span`.
+    /// Creates the token from the given `span`.
     fn new(span: Span) -> Self;
 
-    /// Returns an identifier for this keyword.
+    /// Returns an identifier for this token.
     fn ident(&self) -> Ident;
 
     /// The sequence of punctuations that make up the token.

sway-ast/src/pattern.rs

@@ -15,7 +15,7 @@ pub enum Pattern {
     /// A pattern made of a single ident, which could either be a variable or an enum variant
     AmbiguousSingleIdent(Ident),
     Var {
-        reference: Option<RefToken>,
+        reference: Option<RefToken>, // TODO-IG: Remove this and replace with proper support for references in patterns.
         mutable: Option<MutToken>,
         name: Ident,
     },

sway-ast/src/ty/mod.rs

@@ -22,6 +22,10 @@ pub enum Ty {
         slice_token: SliceToken,
         ty: SquareBrackets<Box<Ty>>,
     },
+    Ref {
+        ampersand_token: AmpersandToken,
+        ty: Box<Ty>,
+    },
 }
 
 impl Spanned for Ty {
@@ -35,6 +39,10 @@ impl Spanned for Ty {
             Ty::Infer { underscore_token } => underscore_token.span(),
             Ty::Ptr { ptr_token, ty } => Span::join(ptr_token.span(), ty.span()),
             Ty::Slice { slice_token, ty } => Span::join(slice_token.span(), ty.span()),
+            Ty::Ref {
+                ampersand_token,
+                ty,
+            } => Span::join(ampersand_token.span(), ty.span()),
         }
     }
 }

sway-core/src/abi_generation/evm_abi.rs

@@ -144,6 +144,9 @@ pub fn abi_str(type_info: &TypeInfo, type_engine: &TypeEngine, decl_engine: &Dec
             name,
             trait_type_id: _,
         } => format!("trait type {}", name),
+        Ref(ty) => {
+            format!("__ref {}", abi_str_type_arg(ty, type_engine, decl_engine)) // TODO-IG: No references in ABIs according to the RFC. Or we want to have them?
+        }
     }
 }
 

sway-core/src/abi_generation/fuel_abi.rs

@@ -850,6 +850,9 @@ impl TypeInfo {
                 name,
                 trait_type_id: _,
             } => format!("trait type {}", name),
+            Ref(ty) => {
+                format!("__ref {}", ty.abi_str(ctx, type_engine, decl_engine)) // TODO-IG: No references in ABIs according to the RFC. Or we want to have them?
+            }
         }
     }
 }

sway-core/src/asm_generation/fuel/data_section.rs

@@ -124,6 +124,9 @@ impl Entry {
                 name,
                 padding,
             ),
+            ConstantValue::Reference(_) => {
+                todo!("Constant references are currently not supported.")
+            } // TODO-IG: Implement.
         }
     }
 

sway-core/src/asm_generation/miden_vm/miden_vm_asm_builder.rs

@@ -658,6 +658,7 @@ impl<'ir, 'eng> MidenVMAsmBuilder<'ir, 'eng> {
             String(_) => todo!(),
             Array(_) => todo!(),
             Struct(_) => todo!(),
+            Reference(_) => todo!(),
         }
     }
 }

sway-core/src/control_flow_analysis/dead_code_analysis.rs

@@ -1858,6 +1858,17 @@ fn connect_expression<'eng: 'cfg, 'cfg>(
             }
             Ok(vec![])
         }
+        Ref(exp) | Deref(exp) => connect_expression(
+            engines,
+            &exp.expression,
+            graph,
+            leaves,
+            exit_node,
+            "",
+            tree_type,
+            exp.span.clone(),
+            options,
+        ),
     }
 }
 

sway-core/src/ir_generation/const_eval.rs

@@ -622,6 +622,12 @@ fn const_eval_typed_expr(
                 }
             }
         }
+        ty::TyExpressionVariant::Ref(_) | ty::TyExpressionVariant::Deref(_) => {
+            return Err(ConstEvalError::CompileError(CompileError::Unimplemented(
+                "Constant references are currently not supported.",
+                expr.span.clone(),
+            )));
+        }
         ty::TyExpressionVariant::Reassignment(_)
         | ty::TyExpressionVariant::FunctionParameter
         | ty::TyExpressionVariant::AsmExpression { .. }

sway-core/src/ir_generation/convert.rs

@@ -14,9 +14,9 @@ use sway_types::{integer_bits::IntegerBits, span::Span};
 pub(super) fn convert_literal_to_value(context: &mut Context, ast_literal: &Literal) -> Value {
     match ast_literal {
         // In Sway for now we don't have `as` casting and for integers which may be implicitly cast
-        // between widths we just emit a warning, and essentially ignore it.  We also assume a
-        // 'Numeric' integer of undetermined width is 'u64`.  The IR would like to be type
-        // consistent and doesn't tolerate mising integers of different width, so for now, until we
+        // between widths we just emit a warning, and essentially ignore it. We also assume a
+        // 'Numeric' integer of undetermined width is 'u64`. The IR would like to be type
+        // consistent and doesn't tolerate missing integers of different width, so for now, until we
         // do introduce explicit `as` casting, all integers are `u64` as far as the IR is
         // concerned.
         //
@@ -158,6 +158,7 @@ fn convert_resolved_type(
         TypeInfo::Alias { ty, .. } => {
             convert_resolved_typeid(type_engine, decl_engine, context, &ty.type_id, span)?
         }
+        TypeInfo::Ref(_) => Type::get_uint64(context),
 
         // Unsupported types which shouldn't exist in the AST after type checking and
         // monomorphisation.

sway-core/src/ir_generation/function.rs

@@ -36,25 +36,25 @@ use std::collections::HashMap;
 ///
 /// This is mostly recursively compiling expressions, as Sway is fairly heavily expression based.
 ///
-/// The rule here is to use compile_expression_to_value() when a value is desired, as opposed to a
+/// The rule here is to use `compile_expression_to_value()` when a value is desired, as opposed to a
 /// pointer. This is most of the time, as we try to be target agnostic and not make assumptions
 /// about which values must be used by reference.
 ///
-/// compile_expression_to_value() will force the result to be a value, by using a temporary if
+/// `compile_expression_to_value()` will force the result to be a value, by using a temporary if
 /// necessary.
 ///
-/// compile_expression_to_ptr() will compile the expression and force it to be a pointer, also by
-/// using a temporary if necessary.  This can be slightly dangerous, if the reference is supposed
+/// `compile_expression_to_ptr()` will compile the expression and force it to be a pointer, also by
+/// using a temporary if necessary. This can be slightly dangerous, if the reference is supposed
 /// to be to a particular value but is accidentally made to a temporary value then mutations or
 /// other side-effects might not be applied in the correct context.
 ///
-/// compile_expression() will compile the expression without forcing anything.  If the expression
+/// `compile_expression()` will compile the expression without forcing anything. If the expression
 /// has a reference type, like getting a struct or an explicit ref arg, it will return a pointer
 /// value, but otherwise will return a value.
 ///
-/// So in general the methods in FnCompiler will return a pointer if they can and will get it be
-/// forced into a value if that is desired.  All the temporary values are manipulated with simple
-/// loads and stores, rather than anything more complicated like mem_copys.
+/// So in general the methods in [FnCompiler] will return a pointer if they can and will get it, be
+/// forced, into a value if that is desired. All the temporary values are manipulated with simple
+/// loads and stores, rather than anything more complicated like `mem_copy`s.
 
 pub(crate) struct FnCompiler<'eng> {
     engines: &'eng Engines,
@@ -558,7 +558,13 @@ impl<'eng> FnCompiler<'eng> {
                 self.compile_reassignment(context, md_mgr, reassignment, span_md_idx)
             }
             ty::TyExpressionVariant::Return(exp) => {
-                self.compile_return_statement(context, md_mgr, exp)
+                self.compile_return(context, md_mgr, exp, span_md_idx)
+            }
+            ty::TyExpressionVariant::Ref(exp) => {
+                self.compile_ref(context, md_mgr, exp, span_md_idx)
+            }
+            ty::TyExpressionVariant::Deref(exp) => {
+                self.compile_deref(context, md_mgr, exp, span_md_idx)
             }
         }
     }
@@ -1103,11 +1109,12 @@ impl<'eng> FnCompiler<'eng> {
         }
     }
 
-    fn compile_return_statement(
+    fn compile_return(
         &mut self,
         context: &mut Context,
         md_mgr: &mut MetadataManager,
         ast_expr: &ty::TyExpression,
+        span_md_idx: Option<MetadataIndex>,
     ) -> Result<Value, CompileError> {
         // Nothing to do if the current block already has a terminator
         if self.current_block.is_terminated(context) {
@@ -1119,7 +1126,6 @@ impl<'eng> FnCompiler<'eng> {
             return Ok(ret_value);
         }
 
-        let span_md_idx = md_mgr.span_to_md(context, &ast_expr.span);
         ret_value
             .get_type(context)
             .map(|ret_ty| {
@@ -1130,12 +1136,99 @@ impl<'eng> FnCompiler<'eng> {
             })
             .ok_or_else(|| {
                 CompileError::Internal(
-                    "Unable to determine type for return statement expression.",
+                    "Unable to determine type for return expression.",
                     ast_expr.span.clone(),
                 )
             })
     }
 
+    fn compile_ref(
+        &mut self,
+        context: &mut Context,
+        md_mgr: &mut MetadataManager,
+        ast_expr: &ty::TyExpression,
+        span_md_idx: Option<MetadataIndex>,
+    ) -> Result<Value, CompileError> {
+        let value = self.compile_expression_to_ptr(context, md_mgr, ast_expr)?;
+
+        if value.is_diverging(context) {
+            return Ok(value);
+        }
+
+        // TODO-IG: Do we need to convert to `u64` here? Can we use `Ptr` directly? Investigate.
+        let int_ty = Type::get_uint64(context);
+        Ok(self
+            .current_block
+            .append(context)
+            .ptr_to_int(value, int_ty)
+            .add_metadatum(context, span_md_idx))
+    }
+
+    fn compile_deref(
+        &mut self,
+        context: &mut Context,
+        md_mgr: &mut MetadataManager,
+        ast_expr: &ty::TyExpression,
+        span_md_idx: Option<MetadataIndex>,
+    ) -> Result<Value, CompileError> {
+        let ref_value = self.compile_expression(context, md_mgr, ast_expr)?;
+
+        if ref_value.is_diverging(context) {
+            return Ok(ref_value);
+        }
+
+        let ptr_as_int = if ref_value
+            .get_type(context)
+            .map_or(false, |ref_value_type| ref_value_type.is_ptr(context))
+        {
+            // We are dereferencing a reference variable and we got a pointer to it.
+            // To get the address the reference is pointing to we need to load the value.
+            self.current_block.append(context).load(ref_value)
+        } else {
+            // The value itself is the address.
+            ref_value
+        };
+
+        let reference_type = self.engines.te().get_unaliased(ast_expr.return_type);
+
+        let referenced_ast_type = match *reference_type {
+            TypeInfo::Ref(ref referenced_type) => Ok(referenced_type.type_id),
+            _ => Err(CompileError::Internal(
+                "Cannot dereference a non-reference expression.",
+                ast_expr.span.clone(),
+            )),
+        }?;
+
+        let referenced_ir_type = convert_resolved_typeid(
+            self.engines.te(),
+            self.engines.de(),
+            context,
+            &referenced_ast_type,
+            &ast_expr.span.clone(),
+        )?;
+
+        let ptr_type = Type::new_ptr(context, referenced_ir_type);
+        let ptr = self
+            .current_block
+            .append(context)
+            .int_to_ptr(ptr_as_int, ptr_type)
+            .add_metadatum(context, span_md_idx);
+
+        let referenced_type = self.engines.te().get_unaliased(referenced_ast_type);
+
+        let result = if referenced_type.is_copy_type() || referenced_type.is_reference_type() {
+            // For non aggregates, we need to return the value.
+            // This means, loading the value the `ptr` is pointing to.
+            self.current_block.append(context).load(ptr)
+        } else {
+            // For aggregates, we access them via pointer, so we just
+            // need to return the `ptr`.
+            ptr
+        };
+
+        Ok(result)
+    }
+
     fn compile_lazy_op(
         &mut self,
         context: &mut Context,
@@ -1905,6 +1998,7 @@ impl<'eng> FnCompiler<'eng> {
         if init_val.is_diverging(context) || body_deterministically_aborts {
             return Ok(Some(init_val));
         }
+
         let mutable = matches!(mutability, ty::VariableMutability::Mutable);
         let local_name = self.lexical_map.insert(name.as_str().to_owned());
         let local_var = self
@@ -2695,18 +2789,24 @@ impl<'eng> FnCompiler<'eng> {
                         .map(|init_expr| {
                             self.compile_expression(context, md_mgr, init_expr)
                                 .map(|init_val| {
+                                    let init_type =
+                                        self.engines.te().get_unaliased(init_expr.return_type);
+
                                     if init_val
                                         .get_type(context)
                                         .map_or(false, |ty| ty.is_ptr(context))
-                                        && self
-                                            .engines
-                                            .te()
-                                            .get_unaliased(init_expr.return_type)
-                                            .is_copy_type()
+                                        && (init_type.is_copy_type()
+                                            || init_type.is_reference_type())
                                     {
-                                        // It's a pointer to a copy type.  We need to derefence it.
+                                        // It's a pointer to a copy type, or a reference behind a pointer. We need to dereference it.
+                                        // We can get a reference behind a pointer if a reference variable is passed to the ASM block.
+                                        // By "reference" we mean th `u64` value that represents the memory address of the referenced
+                                        // value.
                                         self.current_block.append(context).load(init_val)
                                     } else {
+                                        // If we have a direct value (not behind a pointer), we just passe it as the initial value.
+                                        // Note that if the `init_val` is a reference (`u64` representing the memory address) it
+                                        // behaves the same as any other value, we just passe it as the initial value to the register.
                                         init_val
                                     }
                                 })

sway-core/src/language/parsed/expression/mod.rs

@@ -308,6 +308,8 @@ pub enum ExpressionKind {
     Continue,
     Reassignment(ReassignmentExpression),
     Return(Box<Expression>),
+    Ref(Box<Expression>),
+    Deref(Box<Expression>),
 }
 
 #[derive(Debug, Clone)]