SlideShare a Scribd company logo
What the &~#@<!?
(Pointers in Rust)

Jodhpur, India (Dec 2011)
Plan for Today
Recap:
Explicit vs. Automatic Memory Management
More Advanced Managed Memory
Systematic, Explicit Memory Management
Last 15 minutes: Forming Teams for PS3

1
Memory Management Options
Unmanaged (Explicit)
C, C++
Up to programmer to free
objects

Managed (Automatic)
Java, C#, Go, Python, Scheme
Objects are automatically
reclaimed

2
Garbage Collection
Mark and Sweep
Compacting
Generational

Go
3
(Advanced “comic
book” version of GC)
4
5
Mark-and-sweep

about:config / javascript.options.mem.gc_incremental

6
Reference Counting
Each object keeps track of the number of
references to it:
if the reference count reaches 0, the object is
garbage
This is the most “incremental” GC can get!
7
Counting References
{
T x = new T ();
…
y = x;
…
}
8
static int
app1(PyListObject *self, PyObject *v)
{
Py_ssize_t n = PyList_GET_SIZE(self);
assert (v != NULL);
if (n == INT_MAX) {
PyErr_SetString(PyExc_OverflowError,
"cannot add more objects to list");
return -1;
}
if (list_resize(self, n+1) == -1)
return -1;
Py_INCREF(v);
PyList_SET_ITEM(self, n, v);
return 0;
}

Python’s list append implementation

#define _Py_NewReference(op) ( 
(op)->ob_refcnt = 1)
#define Py_INCREF(op) ( 
(op)->ob_refcnt++)
#define Py_DECREF(op) 
if (--(op)->ob_refcnt != 0) 
_Py_CHECK_REFCNT(op) 
else 
_Py_Dealloc((PyObject *)(op))
9
Is Reference Counting Enough?

10
Is Reference Counting Enough?
{
BigObject a = new BigObject();
BigObject b = new BigObject();
a.friend = b;
b.friend = a;

}

11
Memory Management Options
Unmanaged (Explicit)
C, C++
Up to programmer to free
objects

Managed (Automatic)
Java, C#, Go, Python, Scheme
Objects are automatically
reclaimed

Is bounds checking orthogonal to memory management?
12
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
int main(int argc, char **argv) {
char *s = (char *) malloc (1024);
char *t = s - 12;
strcpy(s, "Hello!");
s = NULL;

printf("Reaching s: %sn", t + 12);
long int x = (long int) t + 12;
printf("Reaching s: %sn", (char *) x);
return 0;
}

13
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
int main(int argc, char **argv) {
char *s = (char *) malloc (1024);
char *t = s - 12;
strcpy(s, "Hello!");
s = NULL;

gash> gcc -Wall managed.c
gash>./a.out
Reaching s: Hello!
Reaching s: Hello!

printf("Reaching s: %sn", t + 12);
long int x = (long int) t + 12;
printf("Reaching s: %sn", (char *) x);
return 0;
}

14
PLDI 1996

15
another paper from that conference…
PLDI 1996

16
Complaints about my earlier tool:

comp.os.linux post, August 1994
17
“Willy-Nilly” Memory Management

Systematic Memory Management
18
Static Detection of
Dynamic Memory
Errors, David Evans,
PLDI May 1996
19
20
Note: these are “compile-time” errors (just produced by a separate tool).
21
A box is a reference to a heap allocation holding another value.
There are two kinds of boxes: managed boxes and owned boxes.
An owned box type or value is constructed by the prefix tilde sigil ~.
Rust Manual, Section 9.1.4

let mut gname : ~str = ~"annotations";

22
Moving Pointers
Lose reference of
owned pointer
after it is
transferred.

fn main() {
let owned = ~"All mine!";
println!("Whose is it? {:s}", owned);
let stolen = owned;
println!("Whose is it? {:s}", stolen);
}

23
fn main() {
let owned = ~"All mine!";
let stolen = owned;
println!("Whose is it? {:s}", owned);
} owned.rs:4:34: 4:39 error: use of moved value: `owned`
owned.rs:4 println!("Whose is it? {:s}", owned);
^~~~~
note: in expansion of format_args!
<std-macros>:195:27: 195:81 note: expansion site
<std-macros>:194:5: 196:6 note: in expansion of println!
owned.rs:4:4: 4:41 note: expansion site
owned.rs:3:8: 3:14 note: `owned` moved here because it has type `~str`, which is moved by
default (use `ref` to override)
owned.rs:3 let stolen = owned;
^~~~~~
error: aborting due to previous error
24
fn main() {
let owned = ~"All mine!";
let ref stolen = owned;
println!("Whose is it? {:s}", owned);
println!("Whose is it? {:s}", *stolen);
}
fn main() {
let owned: ~str = ~"Mine, all mine!";
let ref stolen : ~str;
stolen = &owned;

println!("Whose is it? {:s}", *stolen);
}
25
fn main() {
let owned: ~str = ~"Mine, all mine!";
let ref stolen : ~str;
stolen = &owned;
fn main() {
let ref stolen : ~str;
println!("Whose is it? {:s}", *stolen);
}
{
let mine: ~str = ~"Mine, all mine!";
stolen = &mine;
}
println!("Whose is it? {:s}", *stolen);
}

26
lifetimes.rs:6:16: 6:21 error: borrowed
value does not live long enough
lifetimes.rs:6
stolen = &mine;
^~~~~
lifetimes.rs:1:11: 10:2 note: reference
must be valid for the block at 1:10...
...
lifetimes.rs:4:4: 7:5 note: ...but
borrowed value is only valid for the
block at 4:3
…

fn main() {
let ref stolen : ~str;

{
let mine: ~str = ~”Mine!";
stolen = &mine;
}
...
}

See Kiet’s blog to understand more about how the Rust compiler does this:
http://ktt3ja.github.io/blog/2014/02/10/understanding-rusts-lifetime-inference/
27
Object’s Lifetime

We cannot borrow an
object for longer than
that object may live!

Length of “loan”

Borrow Lifetimes

28
fn bigger(s1: &str, s2: &str) -> &str {
if s1.len() > s2.len() { s1 } else { s2 }
}
fn main() {
let s: ~str = ~"Mine!";
let t: ~str = ~"Yours!";
println!("Whose is it? {:s}", bigger(s, t));
}

29
borrow.rs:2:5: 2:46 error: cannot infer an appropriate lifetime due to conflicting requirements
borrow.rs:2 if s1.len() > s2.len() { s1 } else { s2 }
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
borrow.rs:1:39: 3:2 note: first, the lifetime cannot outlive the anonymous lifetime #2 defined on the block at 1:38...
borrow.rs:1 fn bigger(s1: &str, s2: &str) -> &str {
fn bigger(s1: &str, s2: &str) -> &str {
borrow.rs:2 if s1.len() > s2.len() { s1 } else { s2 }
borrow.rs:3 }
if s1.len() > s2.len() { s1 } else { s2 }
borrow.rs:2:5: 2:46 note: ...so that if and else have compatible types (expected `&str` but found `&str`)
}
borrow.rs:2 if s1.len() > s2.len() { s1 } else { s2 }
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
borrow.rs:1:39: 3:2 note: but, the lifetime must be valid for the anonymous lifetime #3 defined on the block at
1:38...
borrow.rs:1 fn bigger(s1: &str, s2: &str) -> &str {
borrow.rs:2 if s1.len() > s2.len() { s1 } else { s2 }
borrow.rs:3 }
borrow.rs:2:5: 2:46 note: ...so that types are compatible (expected `&str` but found `&str`)
borrow.rs:2 if s1.len() > s2.len() { s1 } else { s2 }
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
error: aborting due to previous error
30
fn bigger<'a>(s1: &'a str, s2: &'a str) -> &'a str {
if s1.len() > s2.len() { s1 } else { s2 }
}

Lifetime parameter: Rust infers minimum
lifetime of all uses, and bind it to parameter
31
fn bigger<'a>(s1: &'a str, s2: &'a str) -> &'a str {
if s1.len() > s2.len() { s1 } else { s2 }
}
fn main() {
let s: ~str = ~"Mine!";
let r: &str;
{

let t: ~str = ~"Yours!";
r = bigger(s, t);
}

println!("Whose is bigger? {:s}", r);
}
32
fn bigger<'a>(s1: &'a str, s2: &'a str) -> &'a str {
if s1.len() > s2.len() { s1 } else { s2 }
}
fn main() {
let s: ~str = ~"Mine!";
let r: &str;
{
let t: ~str = ~"Yours!";
r = bigger(s, t);

}

borrow2.rs:11:21: 11:22 error: borrowed value does not live long enough
borrow2.rs:11
r = bigger(s, t);
println!("Whose is bigger? {:s}", r); ^
borrow2.rs:5:11: 15:2 note: reference must be valid for the block at 5:10...
}
borrow2.rs:9:4: 12:5 note: ...but borrowed value is only valid for the block at
9:3
33
Can we do this in Rust?
34
fn set_name(gname:
pname: ~str) {
*gname = pname;
}

,

35
fn set_name(gname : &mut ~str, pname : ~str) {
*gname = pname;
}

fn main() {
let mut gname : ~str = ~"annotations";
println!("gname = {:s}", gname);
set_name(&mut gname, ~"frees");
println!("gname = {:s}", gname);
}
36
fn set_name(gname : &mut ~str, pname : ~str) {
*gname = pname;
}

Why doesn’t Rust complain about the missing free?
37
Frees?
Where we are going,
we don’t need
frees!

38
Memory Management Options
Unmanaged (Explicit)
C, C++
Up to programmer to free
objects

Managed (Automatic)
Java, C#, Go, Python, Scheme
Objects are automatically
reclaimed

Which is Rust?
39
Problem Set 3

40
Forming Teams for PS3
For this problem set, you are required to work in a team of two or three
people (except in cases where you were notified based on your PS2 teamwork
that you should work alone for PS3, or where you make your own successful
argument before February 19 that it is better for you to work alone).
Your team may not be the same as your team for PS2, so you should either (1)
find a new partner to work with for PS3, or
(2) if you want to work with your PS2 partner again you must find one other
person to join your team.
If you do not end up on a well-formed team by the end of class on 18 February,
you should contact me right away.
41

More Related Content

What the &~#@&lt;!? (Pointers in Rust)

  • 1. What the &~#@<!? (Pointers in Rust) Jodhpur, India (Dec 2011)
  • 2. Plan for Today Recap: Explicit vs. Automatic Memory Management More Advanced Managed Memory Systematic, Explicit Memory Management Last 15 minutes: Forming Teams for PS3 1
  • 3. Memory Management Options Unmanaged (Explicit) C, C++ Up to programmer to free objects Managed (Automatic) Java, C#, Go, Python, Scheme Objects are automatically reclaimed 2
  • 4. Garbage Collection Mark and Sweep Compacting Generational Go 3
  • 6. 5
  • 8. Reference Counting Each object keeps track of the number of references to it: if the reference count reaches 0, the object is garbage This is the most “incremental” GC can get! 7
  • 9. Counting References { T x = new T (); … y = x; … } 8
  • 10. static int app1(PyListObject *self, PyObject *v) { Py_ssize_t n = PyList_GET_SIZE(self); assert (v != NULL); if (n == INT_MAX) { PyErr_SetString(PyExc_OverflowError, "cannot add more objects to list"); return -1; } if (list_resize(self, n+1) == -1) return -1; Py_INCREF(v); PyList_SET_ITEM(self, n, v); return 0; } Python’s list append implementation #define _Py_NewReference(op) ( (op)->ob_refcnt = 1) #define Py_INCREF(op) ( (op)->ob_refcnt++) #define Py_DECREF(op) if (--(op)->ob_refcnt != 0) _Py_CHECK_REFCNT(op) else _Py_Dealloc((PyObject *)(op)) 9
  • 11. Is Reference Counting Enough? 10
  • 12. Is Reference Counting Enough? { BigObject a = new BigObject(); BigObject b = new BigObject(); a.friend = b; b.friend = a; } 11
  • 13. Memory Management Options Unmanaged (Explicit) C, C++ Up to programmer to free objects Managed (Automatic) Java, C#, Go, Python, Scheme Objects are automatically reclaimed Is bounds checking orthogonal to memory management? 12
  • 14. #include <stdio.h> #include <stdlib.h> #include <string.h> int main(int argc, char **argv) { char *s = (char *) malloc (1024); char *t = s - 12; strcpy(s, "Hello!"); s = NULL; printf("Reaching s: %sn", t + 12); long int x = (long int) t + 12; printf("Reaching s: %sn", (char *) x); return 0; } 13
  • 15. #include <stdio.h> #include <stdlib.h> #include <string.h> int main(int argc, char **argv) { char *s = (char *) malloc (1024); char *t = s - 12; strcpy(s, "Hello!"); s = NULL; gash> gcc -Wall managed.c gash>./a.out Reaching s: Hello! Reaching s: Hello! printf("Reaching s: %sn", t + 12); long int x = (long int) t + 12; printf("Reaching s: %sn", (char *) x); return 0; } 14
  • 17. another paper from that conference… PLDI 1996 16
  • 18. Complaints about my earlier tool: comp.os.linux post, August 1994 17
  • 20. Static Detection of Dynamic Memory Errors, David Evans, PLDI May 1996 19
  • 21. 20
  • 22. Note: these are “compile-time” errors (just produced by a separate tool). 21
  • 23. A box is a reference to a heap allocation holding another value. There are two kinds of boxes: managed boxes and owned boxes. An owned box type or value is constructed by the prefix tilde sigil ~. Rust Manual, Section 9.1.4 let mut gname : ~str = ~"annotations"; 22
  • 24. Moving Pointers Lose reference of owned pointer after it is transferred. fn main() { let owned = ~"All mine!"; println!("Whose is it? {:s}", owned); let stolen = owned; println!("Whose is it? {:s}", stolen); } 23
  • 25. fn main() { let owned = ~"All mine!"; let stolen = owned; println!("Whose is it? {:s}", owned); } owned.rs:4:34: 4:39 error: use of moved value: `owned` owned.rs:4 println!("Whose is it? {:s}", owned); ^~~~~ note: in expansion of format_args! <std-macros>:195:27: 195:81 note: expansion site <std-macros>:194:5: 196:6 note: in expansion of println! owned.rs:4:4: 4:41 note: expansion site owned.rs:3:8: 3:14 note: `owned` moved here because it has type `~str`, which is moved by default (use `ref` to override) owned.rs:3 let stolen = owned; ^~~~~~ error: aborting due to previous error 24
  • 26. fn main() { let owned = ~"All mine!"; let ref stolen = owned; println!("Whose is it? {:s}", owned); println!("Whose is it? {:s}", *stolen); } fn main() { let owned: ~str = ~"Mine, all mine!"; let ref stolen : ~str; stolen = &owned; println!("Whose is it? {:s}", *stolen); } 25
  • 27. fn main() { let owned: ~str = ~"Mine, all mine!"; let ref stolen : ~str; stolen = &owned; fn main() { let ref stolen : ~str; println!("Whose is it? {:s}", *stolen); } { let mine: ~str = ~"Mine, all mine!"; stolen = &mine; } println!("Whose is it? {:s}", *stolen); } 26
  • 28. lifetimes.rs:6:16: 6:21 error: borrowed value does not live long enough lifetimes.rs:6 stolen = &mine; ^~~~~ lifetimes.rs:1:11: 10:2 note: reference must be valid for the block at 1:10... ... lifetimes.rs:4:4: 7:5 note: ...but borrowed value is only valid for the block at 4:3 … fn main() { let ref stolen : ~str; { let mine: ~str = ~”Mine!"; stolen = &mine; } ... } See Kiet’s blog to understand more about how the Rust compiler does this: http://ktt3ja.github.io/blog/2014/02/10/understanding-rusts-lifetime-inference/ 27
  • 29. Object’s Lifetime We cannot borrow an object for longer than that object may live! Length of “loan” Borrow Lifetimes 28
  • 30. fn bigger(s1: &str, s2: &str) -> &str { if s1.len() > s2.len() { s1 } else { s2 } } fn main() { let s: ~str = ~"Mine!"; let t: ~str = ~"Yours!"; println!("Whose is it? {:s}", bigger(s, t)); } 29
  • 31. borrow.rs:2:5: 2:46 error: cannot infer an appropriate lifetime due to conflicting requirements borrow.rs:2 if s1.len() > s2.len() { s1 } else { s2 } ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ borrow.rs:1:39: 3:2 note: first, the lifetime cannot outlive the anonymous lifetime #2 defined on the block at 1:38... borrow.rs:1 fn bigger(s1: &str, s2: &str) -> &str { fn bigger(s1: &str, s2: &str) -> &str { borrow.rs:2 if s1.len() > s2.len() { s1 } else { s2 } borrow.rs:3 } if s1.len() > s2.len() { s1 } else { s2 } borrow.rs:2:5: 2:46 note: ...so that if and else have compatible types (expected `&str` but found `&str`) } borrow.rs:2 if s1.len() > s2.len() { s1 } else { s2 } ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ borrow.rs:1:39: 3:2 note: but, the lifetime must be valid for the anonymous lifetime #3 defined on the block at 1:38... borrow.rs:1 fn bigger(s1: &str, s2: &str) -> &str { borrow.rs:2 if s1.len() > s2.len() { s1 } else { s2 } borrow.rs:3 } borrow.rs:2:5: 2:46 note: ...so that types are compatible (expected `&str` but found `&str`) borrow.rs:2 if s1.len() > s2.len() { s1 } else { s2 } ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ error: aborting due to previous error 30
  • 32. fn bigger<'a>(s1: &'a str, s2: &'a str) -> &'a str { if s1.len() > s2.len() { s1 } else { s2 } } Lifetime parameter: Rust infers minimum lifetime of all uses, and bind it to parameter 31
  • 33. fn bigger<'a>(s1: &'a str, s2: &'a str) -> &'a str { if s1.len() > s2.len() { s1 } else { s2 } } fn main() { let s: ~str = ~"Mine!"; let r: &str; { let t: ~str = ~"Yours!"; r = bigger(s, t); } println!("Whose is bigger? {:s}", r); } 32
  • 34. fn bigger<'a>(s1: &'a str, s2: &'a str) -> &'a str { if s1.len() > s2.len() { s1 } else { s2 } } fn main() { let s: ~str = ~"Mine!"; let r: &str; { let t: ~str = ~"Yours!"; r = bigger(s, t); } borrow2.rs:11:21: 11:22 error: borrowed value does not live long enough borrow2.rs:11 r = bigger(s, t); println!("Whose is bigger? {:s}", r); ^ borrow2.rs:5:11: 15:2 note: reference must be valid for the block at 5:10... } borrow2.rs:9:4: 12:5 note: ...but borrowed value is only valid for the block at 9:3 33
  • 35. Can we do this in Rust? 34
  • 36. fn set_name(gname: pname: ~str) { *gname = pname; } , 35
  • 37. fn set_name(gname : &mut ~str, pname : ~str) { *gname = pname; } fn main() { let mut gname : ~str = ~"annotations"; println!("gname = {:s}", gname); set_name(&mut gname, ~"frees"); println!("gname = {:s}", gname); } 36
  • 38. fn set_name(gname : &mut ~str, pname : ~str) { *gname = pname; } Why doesn’t Rust complain about the missing free? 37
  • 39. Frees? Where we are going, we don’t need frees! 38
  • 40. Memory Management Options Unmanaged (Explicit) C, C++ Up to programmer to free objects Managed (Automatic) Java, C#, Go, Python, Scheme Objects are automatically reclaimed Which is Rust? 39
  • 42. Forming Teams for PS3 For this problem set, you are required to work in a team of two or three people (except in cases where you were notified based on your PS2 teamwork that you should work alone for PS3, or where you make your own successful argument before February 19 that it is better for you to work alone). Your team may not be the same as your team for PS2, so you should either (1) find a new partner to work with for PS3, or (2) if you want to work with your PS2 partner again you must find one other person to join your team. If you do not end up on a well-formed team by the end of class on 18 February, you should contact me right away. 41