Improve and complete implementation of new/delete

[matthijskooijman]

Since #340, the Arduino core exposes a new header (in addition to the previously used new.h), but it was incomplete, causing code that previously relied on a more complete new header (such as provided by uclibc++) to break. This breakage, and various considerations around improving new/delete were discussed in #287.

This PR fixes this by (almost) completing the new header.

This also prepares for letting new terminate, rather than returning null on allocation failure. The standard dictates an exception should be thrown, but that is not possible on AVR, so terminating is more similar to raising an exception that is never caught (and thus probably more standards compliant). Code that needs to (portably) handle allocation failure can (after merging this PR) already start using the nothrow version of new instead of the regular one. For compatibility reasons, the code to terminate instead of returning null is disabled yet, but can be enabled using a macro now already, and should be enabled by default in the future.

This also makes some related improvements to the __cxa_pure_virtual and __cxa_deleted_virtual.

This PR fixes #287 and fixes #47.

[matthijskooijman]

For testing this, I've used the following sketch (manually varying the constant s, the NEW_TERMINATES_ON_FAILURE macro in the "new" header and whether or not std::terminate was defined in the sketch):

#include <new>

constexpr size_t s = 8000;
struct foo {
  char bar[s];
};

struct C {
  int v;
  C() : v (1234) { Serial.println("    constructor"); Serial.flush(); }
  ~C() { Serial.println("    destructor"); Serial.flush(); }
};

namespace std {
  /* void terminate() {
    Serial.println("    terminate");
    Serial.flush();
  }*/
}

void setup() {
  Serial.begin(115200);
  Serial.println("Starting");
  Serial.flush();

  char* arr;
  foo* f;
  C* c;
  
  /***** nothrow new *****/
  
  // put your setup code here, to run once:
  arr = new (std::nothrow) char[s];
  Serial.print("  arr=0x");
  Serial.println((uintptr_t)arr, HEX);
  Serial.flush();
  delete[] arr;
  
  f = new (std::nothrow) foo();
  Serial.print("  f=0x");
  Serial.println((uintptr_t)f, HEX);
  Serial.flush();
  delete f;

  c = new (std::nothrow) C();
  Serial.print("  c=0x");
  Serial.println((uintptr_t)c, HEX);
  if (c) {
    Serial.print("    c.v=");
    Serial.println(c->v);
  }
  Serial.flush();
  delete c;
  
  /***** regular new *****/
  
  arr = new char[s];
  Serial.print("  arr=0x");
  Serial.println((uintptr_t)arr, HEX);
  Serial.flush();
  delete[] arr;
  
  f = new foo;
  Serial.print("  f=0x");
  Serial.println((uintptr_t)f, HEX);
  Serial.flush();
  delete f;
  
  c = new C();
  Serial.print("  c=0x");
  Serial.println((uintptr_t)c, HEX);
  if (c) {
    Serial.print("    c.v=");
    Serial.println(c->v);
  }
  Serial.flush();
  delete c;

  /***** placement (non-allocating) new *****/
  char buf[sizeof(C)];
  
  c = new (buf) C();
  Serial.print("  c=0x");
  Serial.println((uintptr_t)c, HEX);
  if (c) {
    Serial.print("    c.v=");
    Serial.println(c->v);
  }
  Serial.flush();
  c->~C();

  Serial.println("Done");
  Serial.flush();
}

void loop() {}

[matthijskooijman]

It would be good if we could get his reviewed and merged before 1.8.14, since it fixes a regression in 1.8.13.

[matthijskooijman]

I recently wondered about the change we made to (optionally) call std::terminate() on allocation failures, and had a look at how gcc's libstdc++ implements this (which I think we haven't considered before, we've just been looking at the C++ spec).

Turns out that libstdc++ does a similar thing, except that it does not call std::terminate() (which can be replaced by user code), but instead aborts (using the seemingly undocumented __builtin_abort() function see docs and code). Since other Arduino architectures (like ARM) use libstdc++, this means that replacing std::terminate() to catch allocation errors (which was what I was looking for) would work on AVR only (when NEW_TERMINATES_ON_FAILURE is defined in the core), but would not be portable to other cores.

For future reference, what I think will work portably, is completely replacing operator new (regular and array version) with one that calls malloc and then handles allocation failures there. This should work, since these are (since this PR) declared weak and can be overridden (as required by the C++ spec, and also implemented by libstdc++).

However, reading back my own code, it seems like this PR might have made the function declarations weak, rather than the definitions, causing references to operator new become weak, rather than the definitions... I'm going to try the override approach soon, and will report on the results here.

[matthijskooijman]

However, reading back my own code, it seems like this PR might have made the function declarations weak, rather than the definitions, causing references to operator new become weak, rather than the definitions... I'm going to try the override approach soon, and will report on the results here.

Seems this is correct. However:

• This does not create a "weak reference" as I feared, turns out there is a different weakref attribute for that.
• This does make all implementations (that #include <new>) weak, not just the Arduino-provided one.
• Linking order seems to make sketch-supplied operator new preferred over the Arduino-supplied ones, so that is good (this also works when overriding some and still requiring some other operators from new.cpp).

So the weak attribute is in the wrong place (should be fixed - see #487), but fortunately overriding these operators does still work as expected currently.