Ramblings

C functions that should be avoided (part 3) - scanf

Sun, 04 May 2014 00:00:00 +1000

This is the third post on functions that should be avoided. I covered other functions in previous posts:

scanf

int scanf(const char * restrict format, ...);

scanf is yet another common source of confusion for C novices, especially since it often gets introduced early to demonstrate user input. While it is possible to use scanf safely and properly handle input issues, it is usually unnecessarily complicated compared to other means.

Consider a simple program to ask a user for their name and age. A novice who has just seen scanf be used might be tempted to write something like the following:

#include <stdio.h>

int main(void) {
    char name[80];
    int age;
    printf("Please enter your name: ");
    fflush(stdout);
    scanf("%s", name);
    printf("Please enter your age: ");
    fflush(stdout);
    scanf("%d", &age);
    printf("Your name is \"%s\" and your age is \"%d\"\n", name, age);
    return 0;
}

Sure enough, when you run the program things seem to work okay:

Please enter your name: Chris
Please enter your age: 100
Your name is "Chris" and your age is "100"

There are a number of problems with the program, some of which can be easily addressed:

The first scanf is like gets and cannot prevent a buffer overflow on too long an input.
It won’t work as desired if the user decides to enter a full name (i.e. two or more words).
It won’t work as desired if the user enters something for an age that doesn’t look like an integer.
It has no way of knowing if the integer entered was in range for an int. In fact, if the result cannot be represented by an int, the behavior is undefined¹.

We can fix some of these problems reasonably simply:

#include <stdio.h>
#include <stdlib.h>

int main(void) {
    char name[80];
    int age;
    printf("Please enter your name: ");
    fflush(stdout);
    if (scanf("%79[^\n]", name) != 1) {
        exit(EXIT_FAILURE);
    }
    printf("Please enter your age: ");
    fflush(stdout);
    if (scanf("%d", &age) != 1) {
        exit(EXIT_FAILURE);
    }
    printf("Your name is \"%s\" and your age is \"%d\"\n", name, age);
    return 0;
}

Now we’re checking whether scanf succeeded in converting the items successfully, and have prevented the buffer overflow, and have catered for multiword names. I believe we now only have two issues left:

Still undefined behavior if the result of the %d conversion is out of range, no way around this while still using scanf.
If a name longer than the 80 character limit is entered, the rest of the name will be left on the standard input stream and parsed as though it was part of the age input.

At this point you really have no way of recovering with scanf other than discarding the extraneous input on that line until a newline is encountered. However, the problem is that you don’t know if scanf actually consumed a newline during reading of the name. So if you then try to consume all data up to the newline, this could be the next legitimate line of input (the age).

Therefore, it is best to simply use fgets to read the name line, then fgets followed by strtol to read the age line:

#include <errno.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>

int readline(char *buf, size_t size, FILE *stream) {
    if (fgets(buf, size, stream) == NULL) {
        return EOF;
    }
    char *n = strchr(buf, '\n');
    if (n) {
        *n = '\0';
        // If you got here, a line was read successfully without truncation.
    } else {
        int c;
        while ( (c = fgetc(stream)) != EOF && c != '\n');
        if (c == EOF) {
            return EOF;
        }
        // If you got here, a line was read successfully but was truncated.
    }
    return 0;
}
int main(void) {
    char namebuf[50], agebuf[50];
    printf("Please enter your name: ");
    fflush(stdout);
    if (readline(namebuf, sizeof namebuf, stdin)) {
        exit(EXIT_FAILURE);
    }
    printf("Please enter your age: ");
    fflush(stdout);
    if (readline(agebuf, sizeof agebuf, stdin)) {
        exit(EXIT_FAILURE);
    }
    char *endptr;
    errno = 0;
    long age = strtol(agebuf, &endptr, 0);
    if (errno == ERANGE || *endptr != '\0') {
        fprintf(stderr, "Not a valid number or out of range.\n");
        exit(EXIT_FAILURE);
    }
    printf("Your name is \"%s\" and your age is \"%ld\"\n", namebuf, age);
    return 0;
}

Hopefully this code is now reasonably robust. Truncation can be detected and it can optionally exit at that point.

Furthermore, if you want to request the user retype their input in case its invalid (e.g. entering an invalid age integer) you can easily wrap the above code in loops.

If you are not parsing numbers, using fgets to read a line then sscanf to parse the resulting string is a very reasonable approach. It is scanf that is problematic, not sscanf.

There is a gotcha with scanf where you run into a classic newbie infinite loop problem:

#include <stdio.h>
#include <stdlib.h>

int main(void) {
    int age, ret;
    do {
        printf("Please enter your age: ");
        fflush(stdout);
        if ( (ret = scanf("%d", &age)) != 1) {
            if (ret == EOF) {
                exit(EXIT_FAILURE);
            }
            printf("error: invalid age, try again.\n");
        }
    } while (ret != 1);

    printf("Your age is \"%d\"\n", age);
    return 0;
}

Here we are being diligent by checking the return value of scanf, but what actually happens when the user types a non-integer such as “hello” is that this is left on the stream, so the next time scanf is called with %d, it immediately returns 0 since it still doesn’t see an integer. The solution is to consume characters until the next newline and try again. However, as shown above, just using fgets is easier.

References

scanf undefined behavior on out of range integer — C11 §7.21.6.2p10 ↩

Do not trust random websites about C

Sat, 03 May 2014 00:00:00 +1000

Do not trust random websites about C

Almost every web tutorial/book/blog on C programming is riddled with subtle and not so subtle errors. If you are a beginner then you are, by definition, not qualified to determine the quality of the content. If you were qualified, you wouldn’t need to be reading it.

Yes, that includes this website. However, I try to cite the relevant sections in the C standard, and I welcome corrections. Readers should read the relevant parts of the standard and verify what I’m saying is correct. If it’s not, corrections would be greatly appreciated.

Below are some incorrect examples found on the web. If the errors have been since corrected, it happened after this post was written.

Even large companies that produce compilers can’t get it right

#include <stdio.h>

void swapnum(int *i, int *j) {
  int temp = i;
  i = j;
  j = temp;
}

int main(void) {
  int a = 10;
  int b = 20;

  swapnum(&a, &b);
  printf("A is %d and B is %d\n", a, b);
  return 0;
}

A description then follows it:

When the function swapnum() is called, the actual values of the variables a and b are exchanged because they are passed by reference. The output is:

A is 20 and B is 10

In fact, it won’t swap the variables at all. It even has constraint violations. The first and third lines of swapnum try to implicitly convert pointers and integers, and the seconds line simply assigns the copies of the pointers local to that function. Finally, the description claims they’re passed by reference, when they’re passed by value. C doesn’t have pass by reference. This probably stems from the fact that they’re trying to teach both C and C++ on the same page.

It should read:

void swapnum(int *i, int *j) {
  int temp = *i;
  *i = *j;
  *j = temp;
}

You would think I’m picking on Joe Random Blogger tutorials, but the above comes from IBM’s compiler documentation. It is probably a careless oversight from the author, rather than a lack of knowledge of something fundamental to C. However, it is a clear indication that the author has not bothered to test the code.

I left feedback about this error via IBM’s website on 26 Feb 2014, more than 2 months before this post, no response or fix yet.

Learn C The Hard Way (a far more ironically accurate title than is first apparent)

Learn C The Hard Way is a free and popular online book written in the last few years. Unfortunately, in my opinion, it is carelessly written and full of mistakes. I could write a multipart series of posts just on the errors in this book, but it may not be constructive. I have yet to read all the book, but here are some glaring problems I quickly found.

One chapter is titled “Exercise 17: Heap And Stack Memory Allocation”. If you’ve read my previous post called “There is no stack”, you’d quickly realise why I just knew that this was a red flag. The author writes:

The easiest way to keep this straight is with this mantra: If you didn’t get it from malloc or a function that got it from malloc, then it’s on the stack.

Putting aside the fact that C doesn’t prescribe stacks, the above statement still has several problems. Common C implementations may use a stack to store variables of automatic storage duration, so let’s pretend that “it’s on the stack” means “has automatic storage duration”. Except:

What if it was declared at file scope? Then it has static storage duration.
What if it was declared inside a block with the static keyword? Then it has static storage duration.
What if the C implementation decided to put the variable in a register instead for optimization purposes?

Recall from “There is no stack”, objects with static storage duration exist for the lifetime of the program, why would they be put on a stack?

It would have been far more useful for the author to have just spoken about storage duration instead of pretending that certain things always go on a stack, they don’t.

The author goes on to state:

If you take a pointer to something on the stack, and then pass that or return it from your function, then the function receiving it will “segmentation fault” (segfault) because the actual data will get popped off and disappear. You’ll be pointing at dead space.

This is wrong and misleading in several ways. For beginners, a segmentation fault occurs on some platforms when invalid access to storage occurs. A C implementation needn’t provide this facility¹. Let’s set aside this and talk about implementations that do.

A segmentation fault is a useful thing to have, as it is an immediate cue to a developer or even end user that something is wrong with the program. A user could, at the very least, inform the developer than the program is crashing. With a segmentation fault, one can typically then examine a dump of the program’s current state with a debugger, and try to work out what went wrong. Great! Let’s try this on an implementation that’s using a stack:

#include <stdio.h>
#include <string.h>

char *int2str(int x) {
    char a[100];
    snprintf(a, sizeof a, "%d", x);
    return a;
}

int main(void) {
    puts(int2str(42)); // undefined behavior, using pointer
                       // to object whose lifetime has expired
    return 0;
}

Now let’s compile and run this program:

% clang bad.c -o bad
x.c:7:12: warning: address of stack memory associated with local variable 'a' returned [-Wreturn-stack-address]
    return a;
           ^
1 warning generated.

The clang compiler helpfully told me what was wrong. Now I’m going to cast caution to the wind and run the program with undefined behavior:

% ./bad
 d[s��Zs�S�

The output was garbage, but no segmentation fault. Furthermore, if I then compile the same program using the -O2 flag (enabled some optimization), the program actually outputs 42. As we can see here, undefined behavior is entirely unpredictable, and it is misleading to beginners to claim otherwise. The author of Learn C The Hard Way claims it points at “dead space”, but in one instance it’s pointing at garbage (and didn’t segfault) and in another instance it’s still apparently pointing at the “42” string.

In the same quote, the author seems to think it’s wrong to pass pointers to these variables. The same quote:

If you take a pointer to something on the stack, and then pass that … then the function receiving it will “segmentation fault” … because the actual data will get popped off and disappear.

What, so the following code is bad and will segfault? No, it’s fine.

#include <stdio.h>

int main(void) {
    char msg[] = "hello"; // stack variable according to author
    puts(msg); // passing a pointer to "something on the stack"
    return 0;
}

The author of Learn C The Hard Way has demonstrated an unwillingness to acknowledge or correct errors², so I have given up trying to report them to him.

Conclusion

Please don’t try to learn C from random websites. I’ve listed some good books in the resource page that are widely regarded as explaining C properly and correctly. You will save yourself a lot of time and stress in the long run.

There might well be other good C books, but in my experience the majority aren’t.

References

Segmentation faults (SIGSEGV signal) — C11 §7.14p3 ↩
Deconstructing Zed’s K&R2 Deconstruction (first comment presumably from Zed) ↩

C functions that should be avoided (part 2) - strncpy

Fri, 02 May 2014 00:00:00 +1000

This is the second post on functions that should be avoided. I covered atoi and gets in a previous post.

strncpy

char *strncpy(char * restrict s1, const char * restrict s2, size_t n);

Many beginners think of strncpy as a “safe” or “safer” version of strcpy. Unfortunately, while it does help prevent writing past the end of the destination buffer, it still isn’t particularly safe.

From the standard¹:

The strncpy function copies not more than n characters (characters that follow a null character are not copied) from the array pointed to by s2 to the array pointed to by s1. If copying takes place between objects that overlap, the behavior is undefined.

If the array pointed to by s2 is a string that is shorter than n characters, null characters are appended to the copy in the array pointed to by s1, until n characters in all have been written.

The strncpy function returns the value of s1.

The biggest problem with this function, is that even if you appear to use it correctly, you might not get a string. Consider:

#include <stdio.h>
#include <string.h>

int main(void) {
    char dest[4];
    char *src = "This string will not fit in dest";
    strncpy(dest, src, sizeof dest); // okay, no undefined behavior.
    printf("%s\n", dest); // undefined behavior.
    return 0;
}

Here strncpy fills dest’s four elements with ‘T’, ‘h’, ‘i’, ‘s’, but the problem is that we don’t have a string as it’s not null terminated. Using it with any function that expects a string, such as printf, results in undefined behavior. We could mitigate the undefined behavior by setting the last element of dest to the null character explicitly, but we still have no way of knowing if the copy wasn’t complete, without checking the length of the original source string.

OpenBSD provided a function not found in standard C called strlcpy, which is now available on many systems:

size_t strlcpy(char *dst, const char *src, size_t size);

The advantages of strlcpy are:

The destination is null terminated even if it wasn’t big enough to fit all of the source string.
The return value is the length of the string it tried to copy (the source string length) so you can detect truncation.

The safer code with strlcpy:

#include <stdio.h>
#include <stdlib.h>
#include <string.h>

int main(void) {
    char dest[4];
    char *src = "This string will not fit in dest";
    size_t len = strlcpy(dest, src, sizeof dest);
    if (len >= sizeof dest) {
        // oh no, truncated.
        exit(EXIT_FAILURE);
    }
    printf("%s\n", dest); // guaranteed safe.
    return 0;
}

However, in my opinion, strlcpy isn’t particularly useful in most cases, because you can accomplish the same thing just using standard C:

#include <stdio.h>
#include <stdlib.h>
#include <string.h>

int main(void) {
    char dest[4];
    char *src = "This string will not fit in dest";
    if (strlen(src) >= sizeof dest)
        // oh no, truncated.
        exit(EXIT_FAILURE);
    }
    strcpy(dest, src); // guaranteed safe due to check.
    printf("%s\n", dest); // guaranteed safe.
    return 0;
}

The only reason strlcpy would be useful is if you are detecting truncation but still want to use the destination result anyway.

One final reason not to use strncpy, as if the above isn’t enough, is that strncpy will pad the rest of the destination buffer with nulls. This is usually a complete waste of time:

char dest[640*1024]; // ought to be enough for anybody
strncpy(dest, "hello", sizeof dest);

Here the strncpy function writes “hello” followed by 655354 superfluous null bytes.

It has been demonstrated that strncpy is not suitable for copying strings. However, the one thing it does excel at is copying data into fixed width, zero padded, records.

C11 provides new, safer, library functions called strcpy_s² and strncpy_s³ as part of its optional runtime bounds checking interfaces (Annex K). However, these functions are not currently widely implemented so I will not cover them here.

References

strncpy — C11 §7.24.2.4 ↩
strcpy_s — C11 §K.3.7.1.3 ↩
strncpy_s — C11 §K.3.7.1.4 ↩

C has no string type

Sun, 27 Apr 2014 00:00:00 +1000

There are several common misconceptions surrounding the concept of strings in C, such as thinking that a string is a char array or a char pointer. Strings are commonly stored in char arrays, but there are char arrays that do not contain strings and strings that are not within char arrays. It is up to the programmer to ensure that when they pass a char pointer to a function that works on strings they’re really passing a pointer to string. There is no specific type for strings. You can only say that a particular object contains a string.

The C Programming Language as described in the C standards consists of both the language itself and the “C standard library”. Although strings are mentioned beforehand, the library is described in section 7, and it is there where strings are formally defined¹:

A string is a contiguous sequence of characters terminated by and including the first null character. The term multibyte string is sometimes used instead to emphasize special processing given to multibyte characters contained in the string or to avoid confusion with a wide string. A pointer to a string is a pointer to its initial (lowest addressed) character. The length of a string is the number of bytes preceding the null character and the value of a string is the sequence of the values of the contained characters, in order.

The first sentence completely defines strings. The rest defines other terms associated with strings. Important concepts:

Strings are zero or more characters followed by a null terminator.
The length of a string doesn’t include the null. i.e. strlen(“hello”) is 5.

The following examples are of things that do not contain strings:

// These aren't strings because they
// have no null terminator.
char *a = (char[2]){ 1, 2 }; // doesn't point to a string.
char b[3] = "foo";
char c = 42;
    
// These examples are uninitialized and
// have indeterminate content. Assume they're
// in block scope.
char d[3];        
char *e;

The following examples contain strings:

char *p = "foo";  // p points to a string.
char x = 0;       // x contains a zero length string!
char y[] = "foo"; // y contains a string.
    
char *z = malloc(6); // this allocated storage will
                     // soon contain a string.
if (z) 
    strcpy(z, "hello"); // z points to a string.

Another concept of note is that y above actually contains four strings: “foo” starting at y + 0, “oo” starting at y + 1, “o” starting at y + 2 and “” starting (and ending) at y + 3.

You can therefore pass any of these, implicitly converting the arrays to pointers to their first element where appropriate, to functions expecting pointers to strings. However, if the function will attempt to modify the string, passing p there is invalid; it is undefined behavior to attempt to modify string literals². Perhaps surprisingly, string literals are non-const arrays³, so a compiler needn’t stop you from trying to directly modify them.

References

Definition of string — C11 §7.1.1p1 ↩
Modifying string literals undefined — C11 §6.4.5p7 ↩
String literals are non-const arrays — C11 §6.4.5p6 ↩

C functions that should be avoided (part 1) - atoi and gets

Sun, 27 Apr 2014 00:00:00 +1000

Most non-beginner C programmers by now know not to use gets(). Indeed, it has even been removed completely from the latest C standard. However, there are a few other functions that should be mostly avoided for various reasons.

atoi() and friends

double atof(const char *nptr);
int atoi(const char *nptr);
long int atol(const char *nptr);
long long int atoll(const char *nptr);

It is tempting to use atoi and its counterparts to convert string representations of numbers to appropriate types. For example:

#include <stdio.h>
#include <stdlib.h>

int main(int argc, char *argv[]) {
    if (argc > 1) {
        printf("%d\n", atoi(argv[1]));
    }
    return 0;
}

Looks fine, except:

if you invoke the program with a number that is out of range for int, the behavior is undefined¹
If you call atoi on a non-number, such as atoi(“potato”), you will get 0.
atoi("123abc") will return 123 without any indication that there’s other non-numeric junk remaining.

The solution is to use strtol instead:

#include <stdio.h>
#include <stdlib.h>
    
int main(int argc, char *argv[]) {
    if (argc > 1) {
        printf("%ld\n", strtol(argv[1], NULL, 10);
    }
    return 0;
}

This code is better, as we have eliminated the undefined behavior, but it still suffers from being unable to validate input. The second argument to strtol should be used:

#include <stdio.h>
#include <stdlib.h>
#include <errno.h>
    
int main(int argc, char *argv[]) {
    if (argc < 2) {
        fprintf(stderr, "No command line argument found.\n");
        exit(EXIT_FAILURE);
    }
        
    char *endptr;
    long result = strtol(argv[1], &endptr, 0);
    if (errno == ERANGE || *endptr != '\0') {
        fprintf(stderr, "Not a valid number or out of range.\n");
        exit(EXIT_FAILURE);
    }
    printf("%ld\n", result);
    return 0;
}

With strtol we’re able to verify the input fully. Inputs of “123foo” or “potato” will be treated as invalid, and out of range results will be handled. If you are expecting to parse the string further after the number, you can use endptr to know where to continue parsing.

Don’t use these	Use these instead
atoi	strtol
atol	strtol
atoll	strtoll
atof	strtod

gets()

char *gets(char *s); // from C99, removed in C11.

Almost everyone knows not to use gets(), but I’m including it here for completeness. The problem is that gets() doesn’t know how much room your buffer has:

#include <stdio.h>
    
int main(void) {
    char buffer[20];
    gets(buffer);
    return 0;
}

Obviously, if you supply more than 20 bytes on a line to standard input, you will get a buffer overflow. This is undefined behavior. Thousands of less obvious buffer overflows in real code lead to security exploits. scanf has potentially the same problem, but can be mitigated:

scanf("%s", buffer);   // Same problem as gets
scanf("%40s", buffer); // Okay, as long as you understand
                       // how scanf works. Covered in part 3.

The solution is to use fgets instead:

#include <stdio.h>
#include <stdlib.h>
#include <string.h>
    
int main(void) {
    char buffer[20];
    if (fgets(buffer, sizeof buffer, stdin) == NULL) {
        fprintf(stderr, "end-of-file or error.\n");
        exit(EXIT_FAILURE);
    }
    char *p;
    if ((p = strchr(buffer, '\n')) == NULL) {
        fprintf(stderr, "line too long.\n");
        exit(EXIT_FAILURE);
    }
    *p = '\0'; // often we don't care about the newline anymore.
    printf("Line supplied: \"%s\"\n", buffer);
    return 0;
}

Here we’ve checked the return value of fgets and made sure we read a whole line.

C11 actually also provides gets_s as part of its optional bounds checking extensions. I’m not going to cover it here, as the C implementations I use currently don’t support it. Furthermore, in the gets_s recommend practice, it even says²:

The fgets function allows properly-written programs to safely process input lines too long to store in the result array. In general this requires that callers of fgets pay attention to the presence or absence of a new-line character in the result array. Consider using fgets (along with any needed processing based on new-line characters) instead of gets_s.

I cover strncpy in part 2.

References

atoi undefined when out of range — C99 §7.22.1p1 ↩
gets_s recommend practice — C99 §K.3.5.4.1p6 ↩

Pointers are not integers

Mon, 21 Apr 2014 12:00:00 +1000

Many beginners and even seasoned professional programmers seem to think that pointers are integers on some level. It is counterproductive and incorrect to think this way. The problem probably stems from people either using their favourite compiler and examining the resulting assembly language or having experience with assembly language prior to learning C and then forming preconceived notions about how C works. When someone refers to “the machine” in the context of talking about C, it is a fairly reliable indicator that bullshit is ensuing. My previous post probably be read before this one.

The C standard does allow you to convert between integers and pointers, which already illustrates that they’re not the same thing. Furthermore, there are several issues with converting pointers to integers¹:

Pointer to integer conversions are implementation-defined.
If the result of a pointer to integer conversion is out of range, the behavior is undefined.
There is no requirement for the result of a pointer to integer conversion to be in range for any integer type.

There are also several issues with integer to pointer conversions²:

Integer to pointer conversions are implementation-defined.
The pointer might not be correctly aligned.
The pointer might not point to an entity of the referenced type.
The pointer might be a trap representation.

What this also means is that conversion from pointer to integer and back needn’t result in the original pointer value.

C, since C99, does provide intptr_t and uintptr_t which have the property of being signed and unsigned integers respectively that any valid void * can be converted to them and back again to void * and the result will compare equal to the original pointer. However, these types are optional — an implementation needn’t provide them³.

If pointers are not integers, why can we add to and subtract from them?

You can only do this in certain ways. For example, you cannot add two pointers together (constraint violation)⁴ and you cannot subtract two pointers that don’t both point to elements of the same array object or one past the last element (undefined behavior)⁵.

Pointers in C can be considered indexes into arrays. Even when a pointer is pointing to a non-array object, e.g. an int, pointer arithmetic behaves as though it’s a pointer to an array of length one with the type of the object as its element type⁶, e.g. int[1] in this case.

Incrementing a pointer, p, using p++, simply advances the pointer by one element in the array.

Finally, C uses the term “arithmetic types” for integer and floating types⁷. Arithmetic types and pointer types are collectively called “scalar types”⁸. Many operators require their operands to have arithmetic type and therefore do not work with pointers. These include the unary + and - operators and the multiplicative operators (* / and %). All the bitwise operators require integer types.

References

Conversion from pointers to integers — C11 §6.3.2.3p6 ↩
Conversion from integers to pointers — C11 §6.3.2.3p5 ↩
uintptr_t and intptr_t — C11 §7.20.1.4p1 ↩
Cannot add two pointers — C11 §6.5.6p2 ↩
Undefined behavior pointer subtraction — C11 §6.5.6p9 ↩
Non-array objects treated as arrays — C11 §6.5.6p7 ↩
Arithmetic types — C11 §6.2.5p18 ↩
Scalar types — C11 §6.2.5p21 ↩

There is no stack

Mon, 21 Apr 2014 11:00:00 +1000

Some C programmers often refer to “the stack” or “the machine” as though there is some kind of direct link between C and the computer hardware that a program happens to be running on.

The C programming language does actually target a machine, it’s called the “abstract machine”, C11 §5.1.2.3p1:

The semantic descriptions in this International Standard describe the behavior of an abstract machine in which issues of optimization are irrelevant.

C11 §5.1.2.3p4:

In the abstract machine, all expressions are evaluated as specified by the semantics. An actual implementation need not evaluate part of an expression if it can deduce that its value is not used and that no needed side effects are produced (including any caused by calling a function or accessing a volatile object).

C11 §5.1.2.3p6:

The least requirements on a conforming implementation are:

— Accesses to volatile objects are evaluated strictly according to the rules of the abstract machine.

— At program termination, all data written into files shall be identical to the result that execution of the program according to the abstract semantics would have produced.

— The input and output dynamics of interactive devices shall take place as specified in 7.21.3. The intent of these requirements is that unbuffered or line-buffered output appear as soon as possible, to ensure that prompting messages actually appear prior to a program waiting for input.

This is the observable behavior of the program.

What this all means is that it is a C implementation’s responsibility to simply make a program appear to execute on your particular computer in the same manner as described for the abstract machine. C does not prescribe the use of registers, stacks, assembly, or any other computer specific concept one might imagine. C can be either compiled or interpreted or anything in between. For example, a human who reads any input from somewhere and writes the output somewhere according to the program’s semantics could be considered a C implementation.

Consider this simple example:

int main(void) {
    for (long i = 0; i < 1000000000; i++);
    printf("%d\n", i);
    return 0;
}

A C compiler could generate some code to store i in a register or memory, increment i, check if i is still less than 1 billion, loop back and increment i until i is big enough. Alternatively it can prove that there are no side effects from this loop and simply output “1000000000\n” without bothering to use a loop or an i variable, or even call a printf function at all. Many compilers will do this with optimization enabled.

There are no “stack variables” or “heap variables”

Although it is common for compilers to utilise things such as stacks and heaps, there is nothing gained by referring to variables based on where/how we think they will be stored when using a given compiler and machine. Even if the machine and compiler you’re using normally results in placing variables on a stack, how does this help you? Optimization might cause the variable to be placed in a register instead, or eliminated entirely because it’s unused or the [sub]expressions it’s used in result in constants. There’s no value in imagining what a compiler is going to do. If you are concerned about performance you can profile and then find out where the bottlenecks are.

A correct C term which sometimes corresponds to how variables are stored is “storage duration”. Unsurprisingly, storage duration determines the lifetime of objects. C11 has four storage durations¹:

automatic²
static³
allocated⁴
thread (new in C11, optional)⁵

The explanations below are paraphrased or copied directly from the referred standard sections above.

Automatic storage duration

Automatic storage duration is usually the most common. Variables declared within a block without extern or static and variables declared to be a function parameter have automatic storage duration. These objects’ lifetimes extend until the execution of that block ends. Referring to an object outside its lifetime is undefined behavior.

A common beginner mistake is to return a pointer to an automatic storage object from a function and then refer to that object in the calling code. One approach is to return a pointer to a static object, but this is usually a bad idea — it makes the function nonreentrant:

#include <stdio.h>
#include <string.h>

char *contrived(int n) {
    static char result[16];
    strcpy(result, n ? "true" : "false");
    return result;
}

int main(void) {
    char *t = contrived(1);
    printf("t = %s\n", t);
    char *f = contrived(0);
    printf("t = %s\n", t);
    printf("f = %s\n", f);
    return 0;
}
    
// output:
// t = true
// t = false
// f = false

Here, because we’re always pointing to the same result, the subsequent calls overwrite the static array in the function. A better approach is for the caller to allocate space and pass a pointer to where it wants the result to be written.

Static storage duration

If an object is declared at file scope, or declared static as in result in the above code, and is not declared _Thread_local then it has static storage duration. Its lifetime is the entire execution of the program.

Allocated storage duration

When a call to aligned_alloc, calloc, malloc, or realloc succeeds, it returns a pointer to allocated storage. Its lifetime exists until it is deallocated with free or realloc.

Thread storage duration

If an object is declared _Thread_local (new in C!1, optional), it has thread storage duration. Its lifetime is the entire execution of the thread for which it is created. There is a distinct object per thread, and use of the declared name in an expression refers to the object associated with the thread evaluating the expression.

References

Four storage durations — C11 §6.2.4p1 ↩
Automatic storage duration — C11 §6.2.4p5 and C11 §6.2.2p6 ↩
Static storage duration — C11 §6.2.4p3 ↩
Allocated storage duration — C11 §7.22.3p1 ↩
Thread storage duration — C11 §6.2.4p4 ↩

C is not a “try it and see” language

Sun, 20 Apr 2014 00:00:00 +1000

Unlike some languages, it is ineffective to learn C with trial and error. This is because there can be valid C programs that produce no warnings or errors on some or all implementations but may behave differently depending on the compiler, platform, or any arbitrary or random factors. C has the concepts of undefined, unspecified and implementation-defined behavior.

Undefined behavior

Undefined behavior is defined in C11 §3.4.3p1:

No. Consider the following:

behavior, upon use of a nonportable or erroneous program construct or of erroneous data, for which this International Standard imposes no requirements.

There is then an informative note and example:

NOTE Possible undefined behavior ranges from ignoring the situation completely with unpredictable results, to behaving during translation or program execution in a documented manner characteristic of the environment (with or without the issuance of a diagnostic message), to terminating a translation or execution (with the issuance of a diagnostic message).

EXAMPLE An example of undefined behavior is the behavior on integer overflow.

The C standard lists at least 200 different ways undefined behavior can occur. I’m not going to list them all here. Here are a few examples:

// Accessing a pointer which points to an 
// object whose lifetime has expired
// C11 §6.2.4p2    
char *x = malloc(4); 
free(x); 
printf("%p\n", (void *)x); // undefined behavior
    
// Multiple unsequenced side effects on the same object
// C11 §6.5p2
int x = 42; 
x = x++; // undefined behavior
    
// Pointing a pointer beyond one past the end of an array
// C11 §6.5.6p8
int a[3];
*p = a + 4; // undefined behavior
    
// Attempting to modify a string literal
// C11 §6.4.5p7
char *x = "foo";
x[0] = 'b'; // undefined behavior

When undefined behavior occurs, the C implementation is not required to issue any diagnostic message and it is at liberty to do absolutely anything, including making your program appear to do whatever you intended.

Furthermore, undefined behavior renders the entire program undefined, not just the point where the bad code exists. Consider this example from “Defining the undefinedness of C”¹:

#include <stdio.h>

int main(void) {
    int r = 0, d = 0;
    for (int i = 0; i < 5; i++) {
        printf("%d\n", i);
        r += 5 / d; // divides by zero
    }
    return r;
}

One might think it is reasonable to assume that the printf statement will be executed at least once and output 0. After all, the undefined behavior is on the line after the printf. However, compilers are permitted to assume that undefined behavior never occurs, as if it did occur, it can do whatever it wants. A common optimization strategy would be to move the 5 / d computation to before the loop as d isn’t changed. If 5 / d causes the program to crash, no printfs will be executed.

Unspecified behavior

Unspecified behavior is defined in C11 §3.4.4:

use of an unspecified value, or other behavior where this International Standard provides two or more possibilities and imposes no further requirements on which is chosen in any instance.

There is then an informative example:

EXAMPLE An example of unspecified behavior is the order in which the arguments to a function are evaluated.

The C standard lists only about 50 ways in which unspecified behavior can occur. Here are a couple of examples:

// Whether two string literals result in distinct arrays
// C11 §6.4.5p7
char *a = "foo";
char *b = "foo";
printf("%d\n", a == b); // unspecified behavior
    
// The order of evaluation of subexpressions that don't
// have a sequence point between them.
// C11 §6.5
f() && g(); // f is called before g as && is a sequence point 
x = f() + g(); // unspecified which function is called first

Unlike undefined behavior, unspecified behavior has a finite number of known possibilities. Which one is chosen by the C implementation in a given instance is unspecified.

Implementation-defined behavior

Implementation-defined behavior is defined in C11 §3.4.1:

unspecified behavior where each implementation documents how the choice is made

There is then an informative example:

EXAMPLE An example of implementation-defined behavior is the propagation of the high-order bit when a signed integer is shifted right.

The C standard lists at least 100 ways in which implementation-defined behavior can occur. Here are a few examples:

// Any non-standard alternative declaration of main
// i.e. other than:
//     int main(void)
//     int main(int argc, char *argv[])
// or equivalent.
// C11 §5.1.2.2.1p1
void main(void) { // potentially allowed if the
                  // implementation documents it
    return 0;
} 
    
// The number of bits in a byte
// C11 §3.6 and §5.2.4.2.1p1
printf("%d\n", CHAR_BIT); // implementation-defined,
                              // must be at least 8
                              
// The values of the members of the
// execution character set
// C11 §5.2.1p1
printf("%d\n", 'a'); // implementation-defined
    
// The result of converting a pointer to an integer
// or vice versa
// C11 §6.3.2.3p5 and §6.3.2.3p6
char y, *x = &y;
int i = (int)x; // implementation-defined and undefined
                // if the result is out of range for int

Summary

When programming in C, it’s very important to at least be familiar with the kind of things that invoke undefined, unspecified or implementation-defined behavior. The C standard has a useful informative list toward the end in Annex J:

J.1 Unspecified behavior
J.2 Undefined behavior
J.3 Implementation-defined behavior

References

Ellison, C., & Rosu, G. (2012). Defining the undefinedness of C (pdf). ↩

The confusion between pointers and arrays in C

Sat, 19 Apr 2014 00:00:00 +1000

The characteristics of pointers and arrays in C are a common source of confusion encountered on forums, IRC channels, mailing lists, tutorials, et cetera.

Aren’t arrays and pointers the same thing?

No. Consider the following:

#include <stdio.h>

int main(void) {
    char array[] = "a string literal";
    char *pointer = "another string literal";
    printf("sizeof array is %zu\n", sizeof array);
    printf("sizeof pointer is %zu\n", sizeof pointer);
    return 0;
}

// Example output on a C implementation:
// sizeof array is 17
// sizeof pointer is 8

The sizeof operator works on the types. array has type char[17] (one char for the null terminator and 16 chars for the string length) and therefore must have a size of 17. pointer has type char * so its size is implementation defined – 8 in the above example.

The fundamental difference here is that an array object has elements as part of it. A pointer consists only of a representation of something that can point to objects of that type. It is common, but not universal, that all pointers on a given C implementation are the same size. It is possible, for example, for a char * and a double * to be different sizes.

Note that you also cannot assign to an array:

int main(void) {
    char array[] = "foo";
    char *pointer = "bar";
    array = pointer; // error, can't do that to arrays.
    pointer = array; // fine, pointer would point to the
                     // first element of "foo".
    return 0;
}

If you are allowed to assign an array to a pointer but not a pointer to an array, then isn’t an array just a kind of “constant pointer”?

No. You’re not actually assigning an array to a pointer, either. In a number of contexts in C, an array is treated as a pointer to its first element. In the standard (C11 §6.3.2.1p3):

Except when it is the operand of the sizeof operator, the _Alignof operator, or the unary & operator, or is a string literal used to initialize an array, an expression that has type ‘‘array of type’’ is converted to an expression with type ‘‘pointer to type’’ that points to the initial element of the array object and is not an lvalue. If the array object has register storage class, the behavior is undefined.

Since we’re attempting to assign an array to a pointer, the array is an operand of the assignment operator and therefore is converted to a pointer to its first element so we’re merely assigning a pointer to a pointer. In this case, &array[0].

In the case of the array = pointer; assignment, this is a constraint violation since the left operand of assignment operators must be a modifiable lvalue¹²

Arrays are not “constant pointers” or any kind of pointer. This implicit conversion exists so that you don’t have to write &array[0] all the time. The same applies in every other circumstance except the ones described in the above quote from the standard.

But I thought that void f(char x[]); and void f(char *x); are the same?

Yes, they are³! This is a special case in function parameter lists. The following are all equivalent:

void f(char *x);
void f(char x[]);
void f(char x[42]);

When you use an array like notation in a function parameter list, it is the same as declaring it as the type of a pointer to its first element. In the function call f(array), you cannot pass an array to functions, you are passing a pointer instead.

How about the fact that you can use the a[b] notation with both arrays and pointers?

Indeed. This is because a[b] is defined to be equivalent to *(a + b)⁴. This also uses the rule above about arrays being converted to pointers to their first element in most contexts. So when a is an operand of the additive operator in a + b it gets converted to a pointer to its first element. That pointer is then advanced b elements along the array⁵ and the indirection operator (unary *) yields the object that the pointer points to⁶. Since addition is commutative, one can also write silly things like 3[a] or 5["abcdef"].

lvalues and modifiable lvalues — C11 §6.3.2.1p1 ↩
Assignment operator constraints — C11 §6.5.1.6p2 ↩
Function declarator parameters in array form — C11 §6.7.6.3p7 ↩
a[b] equivalent to *(a + b) — C11 6.5.2.1p2 ↩
Additive operator when used on pointers — C11 §6.5.6p8 ↩
Unary * operator — C11 §6.5.3.2p4 ↩

Ramblings

C functions that should be avoided (part 3) - scanf

scanf

References

Do not trust random websites about C

Do not trust random websites about C

Even large companies that produce compilers can’t get it right

Learn C The Hard Way (a far more ironically accurate title than is first apparent)

Conclusion

References

C functions that should be avoided (part 2) - strncpy

strncpy

C has no string type

References

C functions that should be avoided (part 1) - atoi and gets

atoi() and friends

gets()

References

Pointers are not integers

If pointers are not integers, why can we add to and subtract from them?

References

There is no stack

There are no “stack variables” or “heap variables”

Automatic storage duration

Static storage duration

Allocated storage duration

Thread storage duration

References

C is not a “try it and see” language

Undefined behavior

Unspecified behavior

Implementation-defined behavior

Summary

Further reading

References

The confusion between pointers and arrays in C

Aren’t arrays and pointers the same thing?

If you are allowed to assign an array to a pointer but not a pointer to an array, then isn’t an array just a kind of “constant pointer”?

But I thought that void f(char x[]); and void f(char *x); are the same?

How about the fact that you can use the a[b] notation with both arrays and pointers?