malloc.c: check allocation sizes against PTRDIFF_MAX.

I don't expect this to actually come up in any circumstance, but it
prevents a warning in some versions of gcc that would otherwise arise
from the use of 'int' to compute the input size: if gcc isn't
confident that the int is positive, then it complains that possible
inputs to malloc might be in the region of 2^64 - (small multiple of a
negative 32-bit int).

I would hope malloc would fail in any case on such an input, so
failing a couple of lines earlier makes no important difference.

Annoyingly, stdint.h is missing in my NestedVM build setup (though it
has stdbool.h - it's not _totally_ C90). So I have to check that at
cmake time.

Also, removed the #defines for smalloc and friends from the tree234
test mode. These were needed in the old build system, when
tree234-test was built ad-hoc without being linked against malloc.c.
But now tree234-test links against the same utils library as
everything else, and can use the real smalloc - and doing so prevents
another of these warnings when compiling with -flto.

1 files changed, 11 insertions, 0 deletions

diff --git a/malloc.c b/malloc.c
index 7e5b87e..39bcfac 100644
--- a/malloc.c
+++ b/malloc.c
@@ -2,6 +2,9 @@
  * malloc.c: safe wrappers around malloc, realloc, free, strdup
  */
 
+#ifndef NO_STDINT_H
+#include <stdint.h>
+#endif
 #include <stdlib.h>
 #include <string.h>
 #include "puzzles.h"
@@ -12,6 +15,10 @@
  */
 void *smalloc(size_t size) {
     void *p;
+#ifdef PTRDIFF_MAX
+    if (size > PTRDIFF_MAX)
+	fatal("allocation too large");
+#endif
     p = malloc(size);
     if (!p)
 	fatal("out of memory");
@@ -32,6 +39,10 @@ void sfree(void *p) {
  */
 void *srealloc(void *p, size_t size) {
     void *q;
+#ifdef PTRDIFF_MAX
+    if (size > PTRDIFF_MAX)
+	fatal("allocation too large");
+#endif
     if (p) {
 	q = realloc(p, size);
     } else {