void foo(T& out) - How to fix output parameters

by Jonathan

There are some cases where you need to return a value from a function but cannot use the return value. It happens, for example, in functions where you want to return multiple values at once. While you can pass multiple inputs to a function - the parameters, you cannot pass multiple return values in the same way.

C++ programmers tend to use a good old (lvalue) reference for that. You take a non-const reference as parameter and assign the output to that reference. The caller will pass a variable and upon function completion find the value of the variable changed.

Yet this approach has some problems: For starters, it is not obvious when just looking at the call that the variable is going to be changed. This is the reason that C++ style guides such as the one used by Google recommend using a pointer for that. The caller then has to explicitly pass in the address of the variable, making it explicit.

But with a pointer you can now pass in nullptr, you have to check for that in the function: A pointer where you really mean “reference” does not follow the guidelines I’ve been advocating for.

So is there not a universal solution?

There is, but first we need to understand the full scope of the problem.

Motivation

Update: Disclaimer

This post does not intend to advocate for or against the use of output parameters in general. The motivation is simply here to acknowledge that output parameters are a thing people might want to use. Then I go on and show how they can be implemented in a nicer way.

I do not want to convince you to use output parameters neither do I want to convince you to not use output parameters. I just want to say: If you want to use output parameters, consider using the technique described here, as it is less error prone. But if you don’t want to use output parameters, don’t use them.

Continue reading.

Let’s assume we’re having a function func() where we want to return multiple values:

int func(int in_a, int in_b, int& out);

The name of this function could be divide() and it returns result and rest.

Using an lvalue reference for that allows calling it like so:

int output;
auto result = func(42, 11, output);

This has a couple of problems, though, as already mentioned:

But there is another problem which occurs in the definition of the function. Consider the following functions, that concatenates all strings from a stream. The following definition can lead to a result which might be surprising:

bool read_strings(std::istream& in, std::string& out)
{
    for (std::string tmp; in >> tmp;)
        out += tmp;
    return !out.empty();
}

I get that this example can simply return the std::string.

But because it returns bool, you can use it in some form of loop or similar.

read_strings() concatenates tmp with out by repeatedly calling operator+=(). This only gives the desired results if out was initially empty. The following call

std::string str = "abc";
read_strings(in, str);

will keep the value of abc in str. This might be surprising behavior.

So even when implementing functions with “naive” output parameters, one has to be careful and not accidentally read from it before setting it to a known value. This is in addition to all the problems for the caller.

The solution?

THIS IS NOT THE REAL SOLUTION, DO READ ON!

All of this can be avoided by simply using a struct:

struct func_result
{
    int first_value;
    int second_value;
};

func_result func(int a, int b);

…

auto res = func(42, 11);
std::cout << res.first_value << ' ' << res.second_value << '\n';

Real code would use proper names for, well, everything shown here, but you get the point. We’ve declared a new type for the return type, one that can represent and store the two values. Then we can just return that type at once.

One “could” also use std::pair/std::tuple for that, but because they do not give meaningful names to their members and instead rely on the order, this is error prone. Using those in any form of public, commonly used API is not something anybody would ever, do, am I right?

I wouldn’t write a blog post telling you just “use a struct if you want to return multiple values”. This is also recommended by the C++ Core Guidelines. Furthermore, that’s not a solution all the time:

std::string a, b;
std::cin >> a >> b;

What are you going to do now?!

There are many situations where you cannot use the return type of a function. Operator overloading is the least convincing one, I can also continue with callbacks and other form of functors you pass to something and they call you back.

See the disclaimer above: This is just here to show that the output parameter problem does not always have to do with multiple return values. You can of course always avoid output parameters, if you really want to.

In all those cases, the return type is fixed, so you can’t use a struct.

The solution

In a discussion about the Google style guide and that particular rule about output parameters, I heard someone - jokingly? - suggesting that one should use std::ref(). In case you don’t know, std::ref() returns a std::reference_wrapper<T>, which is a wrapper over a reference pointer where assignment changes the object it refers to. It is implicitly convertible to T&, so you could use it in our original example like so:

int output;
auto result = func(42, 11, std::ref(output));

But the compiler does not enforce that, so it is not as great as it could be. The next step might be to change the parameter type as well. What happens if we change the output parameter to std::reference_wrapper?

int func(int in_a, int in_b, std::reference_wrapper<int> out);

This is not a great idea, because std::reference_wrapper does not behave like references. Assignment rebinds the reference, so we have to use out.get() = … instead of out = …. Furthermore, std::reference_wrapper is still implicitly created from a reference, so we can just pass it in without being obvious in the caller.

But changing the parameter type is a step in the right direction. We just need a new type - output_parameter<T>. This type should have an explicit constructor taking T& which stores a pointer to the object. Then it needs an assignment operator that takes some T and assigns that to the pointer:

template <typename T>
class output_parameter
{
public:
    explicit output_parameter(T& obj)
    : ptr_(&obj) {}

    // disable assignment because it would have totally different semantics then the other operator=
    // also it is really not needed
    output_parameter& operator=(const output_parameter&) = delete;

    output_parameter& operator=(T value)
    {
        *ptr_ = std::move(value);
        return *this;
    }

private:
    T* ptr_;
};

We make it the parameter type and now a caller has to write:

int output;
auto result = func(42, 11, output_parameter<int>(output));

Hm, that might be too verbose. No problem, simply use a helper function:

int output;
auto result = func(42, 11, out(output));

Now it is obvious that output is an output parameter and modified from the call. Furthermore, you cannot pass output directly, so it is enforced by the compiler

Let’s look at the read_strings() implementation again:

bool read_strings(std::istream& in, output_parameter<std::string> out)
{
    std::string result;
    for (std::string tmp; in >> tmp;)
        result += tmp;
    out = std::move(result);
    return !result.empty();
}

Because we can’t use operator+= on out, we have to use a temporary string and move that in: We can’t accidentally read from out. But this implementation has a bug - after the std::move(), result might be empty. So we have to get the result first:

bool read_strings(std::istream& in, output_parameter<std::string> out)
{
    std::string result;
    for (std::string tmp; in >> tmp;)
        result += tmp;
    auto empty = result.empty();
    out = std::move(result);
    return !empty;
}

Granted, that’s verbose.

We want to prevent reading from out before we know the value. If we just add a get_value() function or similar, this is not statically checked. So we need to make the value available only after we have assigned it.

How can we do that?

Simple: just change the return type of the assignment operator. It is T& T::operator=(…) by convention and to allow a = b = c. But our assignment operator does not really behave like a regular assignment operator, so there is no harm in changing that convention. We can thus change the return type: the only disadvantage we have is removing the ability to do a = b = c, but what would the semantics be anyways?

This only works here because output_parameter is not meant to be used in generic code. It’s only use is as a function parameter, where the type is clearly spelled out.

So, let’s change the signature of output_parameter::operator=():

T& operator=(T value)
{
    *ptr_ = std::move(value);
    return *ptr_;
}

I’ve changed the return type to T& and let it return the value. This is exactly what we want: We can get the value, but only after we know that it is in a known state! There is no way to get the value without assigning it because we can only get it, after we’ve assigned it!

With that our implementation of read_strings() can now look like this:

bool read_strings(std::istream& in, output_parameter<std::string> out)
{
    std::string result;
    for (std::string tmp; in >> tmp;)
        result += tmp;
    return !(out = std::move(result)).empty();
}

We call empty() on the result of the assignment operator which is the value of our output type!

But now we have to create two strings and have the cost of a move assign. Can it be improved?

Sure, just change the implementation:

bool read_strings(std::istream& in, output_parameter<std::string> out)
{
    auto& result = (out = "");
    for (std::string tmp; in >> tmp;)
        result += tmp;
    return !result.empty();
}

We assign out to the empty string directly and are then allowed to work with the output parameter. With just this class already, we’ve completely eliminated the bug that would previously occur on:

std::string str = "abc";
read_strings(in, out(str));

Now this bug cannot occur anymore - by type design!

Sometimes you do want to append to an existing string though. In this case you can easily write read_strings(in, std::move(str), out(str)), where read_strings() is: bool read_strings(std::istream& in, std::string initial, output_parameter<std::string> out). The function appends on initial and move assigns that to out. This is more trouble than a simple reference though, but in-out parameters - in public (!) APIs - are rare anyways.

We’ve thus solved two of the problems, the only thing remaining is the default constructor requirement.

Allowing non-default constructible types

We still have to create the variable that will be used as output prior to the function call. This still requires a default constructor or at least some way to initialize the value beforehand. What we want is a way to just create the storage for the object, not the object itself. We need to represent an object that might not be there yet.

If you’re thinking std::optional or - better - type_safe::optional, you’re almost there. An optional is a type that either has a value or none. Yes, this can be used to achieve our goal because it does not require a default constructor and we can easily augment output_parameter so that it can handle an optional.

But this is not quite the abstraction we want.

We do not want to introduce a null state to our variable throughout its entire lifetime. What we want is a variable where the initialization is just delayed, deferred until we can initialize it. But the important point is: once it is initialized, it will stay initialized, we should not be able to un-initialize it again

The answer is an optional with a reduced interface - deferred_construction<T>. Like optional it has a has_value() function to query if it is initialized and value() to return the value. But the fundamental difference is: once has_value() returns true, it will return true for the entire lifetime of the object, so you can safely rely on that.

It can be implemented using my type_safe::optional like so:

template <typename T>
class deferred_construction
{
public:
    deferred_construction() = default; // creates it un-initialized

    deferred_construction(const deferred_construction&) = default;
    deferred_construction(deferred_construction&&) = default;

    ~deferred_construction() = default;

    // see below
    deferred_construction& operator=(const deferred_construction&) = delete;

    // initializes it
    deferred_construction& operator=(T value)
    {
        assert(!has_value()); // see below
        opt_ = std::move(value);
        return *this;
    }

    // + variadic emplace(Args&&... args) to initialize in-place

    bool has_value() const
    {
        return opt_.has_value();
    }

    // + non-const overload
    const T& value() const
    {
        return opt_.value();
    }

private:
    type_safe::optional<T> opt_;
};

The implementation is straightforward, there are just two unusual design decisions.

First, there is no assignment operator. This is required to ensure that it cannot be un-initialized. Otherwise it would allow writing:

deferred_construction<T> obj;
obj = T(…);
obj = deferred_construction<T>();

While we can simply make that assignment a no-op or assert that other has a value if this has a value, I’ve opted for the more drastically approach of removing it.

Then the operator=() that initialises the object requires that it has not been initialized yet. While the optional itself can handle that, I’ve decided to prevent that. The reason is simple: once the value has been initialized, the deferred_construction wrapper is useless, it has done what it should have done. Then you can (and should) use value() directly.

With this in place, it is now simple to augment output_parameter, so that it can also accept a deferred_construction object and handles it. The first assignment of the output parameter should use the assignment of the deferred construction object, but if it is initialized, it should use value() to assign.

Then we can write:

deferred_construction<std::string> output;
read_strings(in, out(output));

And this code behaves exactly like the very first implementation, it is just safer, more obvious and does not require a default constructor.

Conclusion

output_parameter<T> allows “better” output parameters where you can’t accidentally read the value and the call is obvious. Combined with deferred_construction<T> it allows output parameters for types that are not default constructible.

As you’ve probably guessed a more sophisticated implementation of everything can be found in my type_safe library.

This blog post was written for my old blog design and ported over. If there are any issues, please let me know.