Non-nullable type annotation syntax

[munificent]

This is the tracking bug for the real implementation of the postfix ? syntax to mark a type annotation as nullable as part of supporting non-nullable types (#28619). See the "Syntax" section of the proposal for details.

Since we're still only in the prototype phase, consider this bug to be on hold. If we decide to go ahead and ship a real implementation, I'll update this.

Flag

While this is being implemented, to ensure we don't expose users to inconsistency in our tools, it should be put behind a flag named nnbd.

Tracking issues

• Analyzer (Analyzer: Non-nullable type annotation syntax #27232)
• dart2js (Analyzer: Non-nullable type annotation syntax #27232)
• dev_compiler(dev_compiler: Non-nullable type annotation syntax #27234)
• Formatter (Support NNBD syntax dart_style#536)
• Specification (Specification: Non-nullable type annotation syntax #27235)
• VM (VM: Non-nullable type annotation syntax #27236)
• IntelliJ-plugin (IntelliJ-plugin: Non-nullable type annotation syntax #27237)

[crelier]

Just curious, why was '?' chosen? It seems very counter-intuitive to me, since a question mark implies some kind of optionality. In this case, we rather want to express a restriction. A more assertive exclamation mark would be more appropriate, in my opinion, but I did not study possible grammar conflicts, if that is the reason of this choice.

[lrhn]

The question mark means that the type is nullable/optionally null/union of type and Null, and its absence means that the type is just itself.

[donny-dont]

Any chance union types will come along with this? They seem to be natural fits.

[dynaxis]

This seems just an experiment for now. But I'd like to note that there might be needs for addition of APIs to the standard library. For instance, Kotlin has filterNotNull and mapNotNull on nullable sequence or collection, which return one with non-nullable element type. Dart's Stream and collections would be better to have such new APIs. They can definitely be implemented in user codes, but are really clumsy to do so.

[crelier]

@lrhn Oh, I see. The title is misleading and the text does not clarify. It should actually be "Nullable type annotation syntax" instead of "Non-nullable type annotation syntax".

[zoechi]

@tantumizer

"is-not-null" assertion operator.

This is shorter

return x ?? (throw new NullPointerException);

See also #24892

I find x!! a bit cryptic.
If the return type for the function that contains this code is a non-nullable type it should throw in checked mode anyway and make the check and throw redundant.

[lrhn]

@tatumizer

Good questions, some not decided yet.

Optional parameters: If nothing else changes, you'll have to make them nullable by adding a ? to the type. After all, that is their type. We may consider allowing you to make them non-nullable if they have a non-null default value. If we do that, we should probably also change how default parameters work by making an explicit null argument be equivalent to not having the argument at all. There is some design-space here to explore.

For a nullable var, it's not really a problem as we are moving towards Strong Mode where var means to infer the type. You can write dynamic? but since null is already assignable to dynamic, I think dynamic? will just mean the same as dynamic (so we may disallow it entirely, there is no reason to have two ways to write the same thing). Same for Object and Null.

Too many question-marks. Likely true, but they are consistently about null (except for the original ?: conditional operator).

A not null assertion (that is: take nullable value, throw if it's null, otherwise we know the type).
That could just be x as Foo if the type of x is Foo?. We will have to change as to not allow null for non-nullable types, you would then write x as Foo? to get the current behavior.
We are also considering shorthands to quickly coerce a value to an expected type. Maybe x! will check for null if x has type Foo? and it's used in a position where a (non-nullable) Foo is needed. Maybe that's just being too clever by half.

[zoechi]

If you have

String foo = map["foo"];

then ?? throw new NullPointerException(); is redundant because String already is non-nullable and checked mode should throw.

[floitschG]

As a clarification: if we add only ? (meaning "this type is nullable"), then the non-? type must be non-nullable.
This means that String foo = map["foo"] would statically not be allowed, unless we have implicit downcasts for nullable types, too. (It wouldn't be that awkward, since A is pretty much a subtype of A|Null).

If we don't allow downcast assignments, then there must be a way to go from nullable to non-nullable.
We have the following choices:

• as, conditions and ifs promoting the type: String foo = map["foo"] ?? ... basically falls into that category. This requires no change to the language.
• a special operator: String foo = map["foo"]!! which checks for nullability. I guess this would also include !!. as in map["foo"]!!.bar(). As Lasse suggests, this could potentially be more general, coercing more than just nullable types. For example, it could replace the implicit downcast.

@tatumizer: I'm definitely interested in the reasons for why you had to use !! more often than you thought.

[eernstg]

Currently the nullability experiment is about syntax only.

In Patrice Chalin's proposal (
https://github.com/chalin/DEP-non-null/blob/master/doc/dep-non-null-AUTOGENERATED-DO-NOT-EDIT.md),
nullability is a property of a type (T? is essentially a shorthand for T | Null: B.3.1), and Foo<C> would be the generic class Foo instantiated
with an actual type argument C, which is a non-null type, assuming that
C is the name of a class. The type argument of Foo<E> where E is a
type variable of an enclosing generic entity such as a generic class Bar<E> .. could be a nullable type or a non-null type, depending on the
instantiation of that generic class of which this is an instance. For
instance, if we consider an instance of Bar<int?> and Bar contains an
occurrence of Foo<E> then E is a nullable type, namely int?. If you
want to make sure that a given type is non-null then you may or may not
have an operator for it: For instance, Foo<E!> could be a Foo of
non-null E, which would in this case be int (int? with the ?
stripped off).

When a type variable can stand for a nullable as well as a non-null type it
is necessary to be a little bit smarter in code generation, such that it
will work in both cases, with good performance.

Because of complexities like this, there are quite a number of issues that
we haven't decided on, so we can't promise anything specific about these
design choices. But currently we won't even promise that there will be
anything like a ! operator for stripping ? off of a given type, we are
just looking for a syntax that will work well for ?.

On Tue, Sep 6, 2016 at 3:14 AM, Tatumizer [email protected] wrote:

Another question: consider generic class Foo. What is the meaning of E?
Is it a nullable type? Or only non-nullable type? If the latter, we would
need Foo<E?> to denote nullable E, but this idea is probably not tenable.
So E is nullable. Suppose we want to say that for Foo, we accept generic
type parameter E only if NOT nullable? How to write this restriction? Maybe
Foo<E!!> :) Or what?

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
#27231 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/AJKXUhdiciCSV8iIfIi2pqtBdS5pIByZks5qnL5kgaJpZM4J0Ezv
.

Erik Ernst - Google Danmark ApS
Skt Petri Passage 5, 2 sal, 1165 København K, Denmark
CVR no. 28866984

[munificent]

Replying to a few random things that weren't already covered:

Just curious, why was '?' chosen?

It's the same syntax used to represent nullable types in C#, Ceylon, Fantom, Kotlin, and Swift.

Any chance union types will come along with this?

We are interested in exploring union types too, but they're a big feature with a lot of consequences, so we aren't working on them right now. There's only so many hours in the day. :)

It should actually be "Nullable type annotation syntax" instead of "Non-nullable type annotation syntax".

I considered that, but the obvious response is that Dart 1.0 already has nullable type annotations—all type annotations are nullable. So this is really about adding a way to express non-nullable types. And the way we do that is by adding new syntax for nullable types and changing the existing syntax to mean non-nullable.

I find x!! a bit cryptic.

Me too, but something along these lines might be worth doing. A big part of why we are doing an experiment around non-nullable types is to get answers to usability questions like this. How often do users need to assert that they know something isn't null when the compiler doesn't? We're hoping to implement enough of the static checking to be able to answer that confidently.

String foo = map["foo"]!!;  // I'm sure it's not null, I've just assigned it!

Another part of this experiment is determining how we need to change our core libraries to make it pleasant to work with non-nullable code. In this case, I think Map should support two accessor methods. One returns V? and returns null if the key isn't present. The other returns V and throws if the key isn't found. In this case, you'd use the latter and wouldn't need !!.

There's some interesting API design questions about which of those operations should be [] versus a named method, which is more commonly used, etc. but we need to start trying things out to get a feel for that.

Opened dart sdk code at random place, and in 30 sec got a first example (https://github.com/dart-lang/sdk/blob/master/sdk/lib/collection/linked_list.dart)

That entire class was designed around the idea that E is nullable. Once that assumption is no longer true, there are probably systemic changes you could make to the entire class so that you don't need to sprinkle !! everywhere.

It's also probably true that core low-level collection classes like this will bear the brunt of the manual null checking. They are closer to the metal and need to do things a little more manually. Higher-level application code should hopefully be able to use non-nullable types more easily.

Another question: consider generic class Foo. What is the meaning of E?

I have an answer in mind for this, which I think lines up with Patrice's proposal, but I haven't verified that or written mine down in detail yet. (That's why this issue is about non-nullable syntax. :) ).

The short answer is that here, since you have no constraint, E can be instantiated with either a nullable or non-nullable type. If E was constrained to some non-nullable type, it could only be instantiated with a non-nullable type.

Suppose we want to say that in Foo, E must be NOT nullable. How to write this restriction? Maybe Foo<E!!>

If you want to say E is some specific type, you can give it a constraint and that implicitly constrains it to be non-nullable too, unless the constraint is nullable:

class Point<T extends num> {
  T x, y; // <-- These are non-nullable.
  Point(this.x, this.y);
}

new Point<int>(); // Fine.
new Point<int?>(); // Error! Constraint is non-nullable.

class Pointish<T extends num?> {
  T x, y; // <-- These may or may not be nullable.
  Pointish(this.x, this.y);
}

new Pointish<int>(); // Fine.
new Pointish<int?>(); // Also fine.

I don't currently plan to support a constraint that says, "The type must be non-nullable, but I don't care anything else about the type." I could be wrong, but it doesn't seem very useful to me.

[eernstg]

With the approach where T? stands for T | Null, there is no difference between Object and Object?. This is because the set of objects typeable as Object already includes the object typable as Null.

You could invent an operator computing a "type difference" (think set difference), say \, and then you could express "everything except null" as Object \ Null. You might even claim that the ! operator that we might introduce in order to "strip the ? off" of a given type variable could be defined to compute exactly that (you could say that it stands for the type-level function (Type T) => T \ Null). You could also consider one of the other interpretations of nullability in the first place. But, as mentioned, we're currently focusing on finding a good syntax, so it'll be a while before those other topics come up.

[munificent]

You want to change collection classes to only accept non-nullable type parameters?

No, the collections don't have constraints and should accept nullable type arguments.

My point was that the implementation of LinkedList takes for granted a type system where E is always nullable and non-nullability is not expressible. Once that assumption is no longer true, the implementer of LinkedList, might change how it's implemented. Maybe something like:

class LinkedList<E extends LinkedListEntry<E>>
    extends Iterable<E> {
  E? _first;

  E get first {
    if (_first != null) return _first as E;
    throw new StateError('No such element');
  }

  E get last {
    return first._previous;
  }

  // ...
}

And my claim was that the situation is typical, you need to write '!!' quite often.

Yeah, you may be right, though I'd like to see how it actually works out in practice in the context of Dart and it's core libraries.

[floitschG]

Currently, it would need to be int? index, since it's initialized with null.

There are other similar cases where the null initialization would be annoying:

int x;
if (someBool) {
  x = 499;
} else {
  x = 42;
}

If these cases are too common we will investigate definite-assignment strategies, like in Java, and see if they would fix that problem.

[munificent]

That is, declaration int index; is treated like int? index;

We could do that, but I think it's probably better to be explicit here at the expense of a little verbosity. I think it would be a bad thing if adding an initializer to a variable changed how its type annotation was interpreted.

Otherwise, with these question marks everywhere, program will look like a crossword puzzle or something.

The last time I poked around a corpus, I found that ~90% of variables were non-nullable, so I don't think the ? will be too common. But we won't know for sure until we try, which is what this experiment is about.

[zoechi]

@tatumizer I worked several years in a language that didn't have null (An ERP system - mostly a database frontend - this might be different for code closer to the metal) and I only missed null when I had to deal with outside code (COM components or similar, but there was some special value provided that was converted to null when passed to outside code).

I liked it a lot to not have to deal with null at all.

I think this is the main point of NNBD that null is rarely needed, otherwise we could keep the current NBD.

A way would be (I think it was mentioned somewhere above) to define default values.

int index;

could be treated like

int index = 0;

This would be similar to now where it is treated like

int index = null;

For String it could be the empty string and for classes an instance created by the default constructor or a named constructor MyClass.default().
But I think just now allowing non-nullable types without initialization would be fine as well.

[zoechi]

It seems currently dart is on its path to require fake nullables. In other words, programmer has to lie about the types, just to keep compiler happy.

Why do you think that?

int x;

I think if there is not a defined default value (like 0), this can and should not be allowed.

[munificent]

The very fact that you had to introduce so many syntax rules to cover nullables is the admission that nullable types are not really types.

No, all it shows is that before now, Dart had no syntactic distinction between types and classes. There are places in the language where you must refer to a class. When you're specifying a superclass, superinterface, etc., you need to refer to some class declaration.

Before this proposal, the syntax for types was indistinguishable from the syntax for referring to a class, because there were no type notations that didn't look like classes. As soon as you add ? for non-nullable types, explicit function type syntax, union types, tuples, or any of the other panoply of possible non-nominal kinds of types, we would end up having to split the grammar in this way.

It's just that up to now, we didn't need to, so we could have type in the grammar do double duty as a type and a reference to a class.

[chalin]

@eernstg @munificent: no worries; the original proposal is a bit lengthy, but nullity (as we all know) can be tricky to do "right" in the context of Dart.

All: thanks for keeping the discussions moving forward.

[munificent]

@munificent : is this a valid syntax:

var map=new Map<String?, String>();

Yup, that's fine.

Assuming the answer is "yes": suppose I want to implement my own map that only works with String keys, but I want null to be a valid key. How to write definition of this generic class:

class StringMap<K extends WHAT???, V> {...}

Well, in this case, the answer would be just class StringMap<V> { ... }. String is a sealed type, so there's no reason to make it generic on the key type. The only valid type argument would be String. :)

But let's say you want to define a number set that can be used with ints or doubles and you also want null to be a valid member. You would do:

class NumSet<T extends num> {
  void add(T? value) { ... }
  void remove(T? value) { ... }
  // etc...
}

[munificent]

I made it generic precisely to show that it can handle null keys - so that it can be instantiated with both String and String?

Ah, sorry. I thought you meant you wanted it to always support null keys, regardless of the type parameter type. In my NumSet example, you can not do NumSet<int?> because the constraint is num, which is non-nullable. If you want to allow that, you'd do:

class NumSet<T extends num?> { ... }

This means that in the body of NumSet, T is now a nullable num type, so before you can call methods on it, you have to test for null first.

[munificent]

But why?

The other two would be:

class Sorter<T extends Ordered<T>?> {}
class Sorter<T extends Ordered<T?>> {}
class Sorter<T extends Ordered<T?>?> {}

The ? was in the wrong place. (Also, I think when I first commented I may have forgotten that our tentative plan does allow ? in constraints.)

[munificent]

class MyGenericClass<T extends Object> {...}
class MyGenericClass<T extends Object?> {...}

These would be equivalent since Null is a subtype of Object. Null|Object collapses to Object.

(The latter can be instantiated with any concrete type, but the former - only with non-nullable concrete types)

My current idea for the semantics does not give you a way to express "any type, but not nullable". If the only thing you know about the type parameter is that it's Object, it's not the end of the world to permit null—it supports all of the methods that Object does. It is an object.

All of these questions would be answered by a proposal for the semantics, which I have not yet written down. This issue is just for the syntax.

Do you think we can table this discussion until I have a real proposal to go on? Right now, we're sort of doing a breadth-first traversal through the semantics one comment at a time, which isn't an efficient use of either of our time.

[eernstg]

If we wish to support a distinction between Object with and without null we can remodel the type hierarchy such that Null is not a subtype of Object any more, but we haven't decided that this is a good idea.

[donny-dont]

@munificent the link to the dart2js bug is wrong up top. Off by one error 😉

[osdiab]

Is this not happening? :/ I find this to be a really useful concept for ensuring type safety of my code, and along with the lack of union types is making me wary of switching from TypeScript/React Native to Dart/Flutter in future projects.

[munificent]

It's not happening yet. Moving Dart to strong mode is a necessary pre-condition for non-nullable types. We are doing that in Dart 2. We hoped to get non-nullable types in at the same time, but it proved to be too big of a change to fit into 2.0, so it's going to have to wait until a later version.

[roman-vanesyan]

So now as Dart 2.0 is released would be there any changes so it finally get happened?

[munificent]

Now all that's left is to design and implement non-nullable types, figure out a migration plan, add language features to make them more usable, etc. :) Basically, we have to do all the work. It's a giant feature.

[gbaldeck]

If this year there was only one feature that dart shipped, this would be my choice. I really hope it becomes a high priority soon.

[mraleph]

@gbaldeck and that's exactly what is happening this year see dart-lang/language#110 :)

@munificent Should we close this and all associated issues and indicate that people can follow along the new language process in the language repo?

[munificent]

Yeah, good call. Closing this in favor of dart-lang/language#110 which is the main tracking issue for the current plan to add non-nullable types.