Signed-off-by: Riku Voipio <riku.voipio@linaro.org>

Fix chunk_record() to unlock chunks_mtx before deallocating a base
node, in order to avoid potential deadlock.  This fix addresses the
second of two similar bugs.

Fix chunk_record() to unlock chunks_mtx before deallocating a base node,
in order to avoid potential deadlock.

Reported by Tudor Bosman.

Clarify that malloc_conf is intended only for compile-time
configuration, since jemalloc may be initialized before main() is
entered.

Add no-op bodies to VALGRIND_*() macro stubs so that they can be used in
contexts like the following without generating a compiler warning about
the 'if' statement having an empty body:

	if (config_valgrind)
		VALGRIND_MAKE_MEM_UNDEFINED(ret, size);

Checking for __s390x__ means you work on s390x, but not s390 (32bit)
systems.  So use __s390__ which works for both.

With this, `make check` passes on s390.

Signed-off-by: Mike Frysinger <vapier@gentoo.org>

Fix a locking order bug that could cause deadlock during fork if heap
profiling were enabled.

Fix Valgrind integration to annotate all internally allocated memory in
a way that keeps Valgrind happy about internal data structure access.

Fix a chunk recycling bug that could cause the allocator to lose track
of whether a chunk was zeroed.  On FreeBSD, NetBSD, and OS X, it could
cause corruption if allocating via sbrk(2) (unlikely unless running with
the "dss:primary" option specified).  This was completely harmless on
Linux unless using mlockall(2) (and unlikely even then, unless the
--disable-munmap configure option or the "dss:primary" option was
specified).  This regression was introduced in 3.1.0 by the
mlockall(2)/madvise(2) interaction fix.

Internal reallocation of the quarantined object array leaked the old array.

Reallocation failure for internal reallocation of the quarantined object
array (very unlikely) resulted in memory corruption.

Avoid writing to uninitialized TLS as a side effect of deallocation.
Initializing TLS during deallocation is unsafe because it is possible
that a thread never did any allocation, and that TLS has already been
deallocated by the threads library, resulting in write-after-free
corruption.  These fixes affect prof_tdata and quarantine; all other
uses of TLS are already safe, whether intentionally (as for tcache) or
unintentionally (as for arenas).

Specify both inline and __attribute__((always_inline)), in order to
avoid warnings when using newer versions of gcc.

Revert refactoring of opt_abort and opt_junk declarations.  clang
accepts the config_*-based declarations (and generates correct code),
but gcc complains with:

  error: initializer element is not constant

Submitted by Frederik Deweerdt.

Convert a couple of stragglers from JEMALLOC_* to use config_*.

Update hash from MurmurHash2 to MurmurHash3, primarily because the
latter generates 128 bits in a single call for no extra cost, which
simplifies integration with cuckoo hashing.

Fix AC_PATH_PROG() calls to specify 'false' as the default, so that if
the configure script fails to find a program, the false program is
instead called, and an error occurs.  Prior to this fix, if xsltproc
could not be found, make would not report an error due to the leading -o
in the xsltproc invocation.

Reported by David Reiss.

Add JEMALLOC_ALWAYS_INLINE and use it to guarantee that the entire fast
paths of the primary allocation/deallocation functions are inlined.

Tighten valgrind integration such that immediately after memory is
validated or zeroed, valgrind is told to forget the memory's 'defined'
state.  The only place newly allocated memory should be left marked as
'defined' is in the public functions (e.g. calloc() and realloc()).

Move validation of supposedly zeroed pages from chunk_alloc() to
chunk_recycle().  There is little point to validating newly mapped
memory returned by chunk_alloc_mmap(), and memory that comes from sbrk()
is explicitly zeroed, so there is little risk to assuming that
chunk_alloc_dss() actually does the zeroing properly.

This relaxation of validation can make a big difference to application
startup time and overall system usage on platforms that use jemalloc as
the system allocator (namely FreeBSD).

Submitted by Ian Lepore <ian@FreeBSD.org>.

Linux uses alloca.h; many other operating systems define alloca(3) in
stdlib.h.

Signed-off-by: Garrett Cooper <yanegomi@gmail.com>

In particular:
- ffsl always returns int, not long, on FreeBSD, Linux, and OSX.
- Mute compiler warnings about rv being unused (and the potential for
   compilers optimizing out the call completely) by dumping the value
   with printf(3).

Signed-off-by: Garrett Cooper <yanegomi@gmail.com>

This ensures POLA on FreeBSD (at least) as free(3) is generally assumed
to not fiddle around with errno.

Signed-off-by: Garrett Cooper <yanegomi@gmail.com>

Modify processing of the lg_chunk option so that it clips an
out-of-range input to the edge of the valid range.  This makes it
possible to request the minimum possible chunk size without intimate
knowledge of allocator internals.

Submitted by Ian Lepore (see FreeBSD PR bin/174641).

Fix chunk_recycyle() to unconditionally inform Valgrind that returned
memory is undefined.  This fixes Valgrind warnings that would result
from a huge allocation being freed, then recycled for use as an arena
chunk.  The arena code would write metadata to the chunk header, and
Valgrind would consider these invalid writes.

Reported by Mike Hommey.

When statically linking jemalloc, it may be beneficial not to export its
symbols if it makes sense, which allows the compiler and the linker to do
some further optimizations.

Refactor arena_prof_accum() and its callers to avoid arena locking when
prof_interval is 0 (as when profiling is disabled).

Reported by Ben Maurer.

Tweak chunk purge order to purge unfragmented chunks from high to low
memory.  This facilitates dirty run reuse.

Based on http://www.canonware.com/pipermail/jemalloc-discuss/2012-March/000164.html