<Complex> and Quaternions/Vectors

[BraedonWooding]

I think currently one of the big places Zig could be really useful is in data science, for this to occur Zig needs damn good math functionality including stuff like Quaternions and Vectors along with some good complex number support (quaternions are 4 tuple complex numbers if you don't know, furthermore quaternions do need good euler angle support for game dev).

I currently have a brief hackish solution for vectors, which currently only supports a single 3D vector though of course that means it supports 2D vectors (and has a few optimisations in place for those cases such as for getting the angle or doing a cross product). Currently I implemented a quick invsqrt with a doom like fast inv sqrt which is more for game development where you only care about 1-2 decimal points of precision in some cases.

You can find it here.

Anyways at minimum I'm going to make a library that holds them for my stuff, but I think they would be very valuable for the std lib.

[fithisux]

I also believe that Zig can shine in scientific computing in general. There is a lot of code abandoned or written once and forgotten in universities or by post-grads for research purposes that carries a lot of value but is written in C or C++(in the C with objects style) or desperately slow Java in a very clumsy manner or after a fierce fight with compiler.

Zig can help here to do an incremental port from C-whatever to Zig and apply proper development best practices (without spending time to rewrite the code from scratch or manually transpile). There are other languages out there but Zig can focus on this area, from proteins, genomics to computational fluid dynamics or climate science. Computing is not only or webapps or mobileapps (people got it wrong).

[BraedonWooding]

Completely agree that we need to deeply consider scientific computing! I think however the problem is a two phase issue, the first is to provide some nice utilities that are standardised around handling things like 'bigInts', vectors, matrices and so on. Then have a separate library (probably not official) to handle the more specific needs of scientific computing such as handling equations and the general use 'maple/matlab/whatever' that academics use (well not just academics also data scientists these days).

We have to be careful that we don't develop a bloated std like in C++, and that we take a more conservative approach like in C/go (but don't go too far and be too inconsistent like C, where for some reason quick sort exists but almost anything else you have to build yourself). So I think having a well done mathematical library with things like Sedenions and Octonions should be separate to the std kinda like how numpy and scipy are done with python, however I do think that quite a few features in numpy like multidimensional arrays should be in python, but others are just not needed, especially around the region of graphing.

[fithisux]

@BraedonWooding : You have it right. Keep Zig a core part and make everything a library. Community should be able to build everything with Zig. The only requirements of Zig is ergonomics and enablement, no bloat not swiss army knife in the true spirit of C and Scheme.

[lerno]

Just a note here: if there are built in fundamental types for n-dimensional vectors, complex and quarternion numbers + matrices, then we could have built-in overloading for those types, allowing basic operations to be expressed clearly, e.g. a * b + c instead of add(mult(a, b), c).

[isaachier]

@lerno I agree that might make more sense, but my own preference would be to keep fundamental types to those exposed by the LLVM IR. Otherwise, they are just additional features that need not be in the compiler. Furthermore, you might consider allowing operator overloads (like C++ and Ruby) if you prefer that syntax.

[lerno]

I assumed operator overloading was out for Zig?

[ghost]

I assumed operator overloading was out for Zig?

yes this is why, as I understand, he argues that your proposal is a contradiction to the current design decision

IF your proposal was accepted then zig would also need/ have op overloading in general because your proposal implies that this is a useful feature/ good tradeoff

[lerno]

From what I understand C is doing some form of overloading too in its support of complex numbers. The use case is not general operator overloading but specific overloading for certain basic math operations. General operator overloading is a bad idea.

[ghost]

Thats just super weird IMO, I did not know this but you're definitely right that its a "feature".
But c also has "generics" ...

IMO if zig would acknowledge that op overloading is a useful feature, it should go all the way and make it accessible to all users (what about my own finite field arithmetic?) not only compiler supported types. Thus I don't think this fits onto zig anymore.

Maybe one possible option might be to allow op overloading for * / + - ^ or some subset of operations.

[isaachier]

I think the issue with operator overloading is that it is too prone to abuse. So as I said originally, it is probably best to avoid relying on operators unless the type is fundamental. When it comes to fundamentals, the LLVM IR is probably the best way to decide.

[binary132]

It seems like a lot of people want various built-in features that, while interesting, don't seem to fit with the spirit of Zig as a lean, mean C competitor. I sometimes wonder if the right way to let them all get what they want is to support language extension via third-party package, a bit like GHC's LANGUAGE pragma, but with an API that can support third-party extension and resolution of third-party LANGUAGE packages.

I certainly don't care for built-in operator-overloaded linear algebraic features, and I would prefer not to waste the time of core developers with many nice-to-have features. But it would be awesome if there way a way for interested parties to develop syntax packages. @andrewrk any feedback?

[kyle-github]

One problem with operator overloading, even for things like matrices, is that you so easily hide what is going on. C++ takes this to an extreme where just seeing "a + b" gives you no real idea of what is going on. Matrix addition? String concatenation? Set union?

Since Zig already started down the path of having + and +% and so on, perhaps there is a way to provide for some amount of operator overloading to make these things a little nicer on the eyes but still keep basic operations clear. Perhaps names like "_+_" could be overloaded or something (do not take that particular syntax as a real proposal). Then matrices could be added with "a _+_ b".

Not a serious proposal, but perhaps there is some happy medium that allows external packages to provide almost the same syntax as built in things but still allow a very clear indication of when some operation needed more investigation before you automatically assume what it does. If I see "add()", I know it is a function and I need to go look up what it does. If I see "+", I have to know the types of both arguments in C++ and even then this gets hard with all the automatic coercion that goes on. If I saw "_+_" I could treat it like "add()" and make the effort to find out what it really calls.

[phase]

In Why Zig When There is Already CPP, D, and Rust?, Zig prides itself for not having hidden control flow, which I believe is a good decision. Something that can help users who want operator overloading but don't want the hidden function calls would be explicit overloaded operators.

@ could be used as a prefix for operators that have been overloaded.

var c = a @+ b;

This tells the user that there are hidden functions being called here, but we are explicitly stating they are. Allowing structures to override these can make using libraries like a 3rd party Bignum library usable without being painful.

[0joshuaolson1]

I used to be a fan of mixfix operators (where overloaded + is a more common example). One issue is how to explicitly communicate what module the operator function came from:

a matrix_math.+ b;

is hardly better than

matrix_math.add(a, b)

and can't be used like a normal + if you'd like to do

matrix_math.add(a, b, c, d)

without horrible verbosity.

[lerno]

I think that @kyle-github raises a very important thing: overloaded operators end up being very confusing because they look fundamental yet they are not. I fully agree with that being a problem.

Also, overloading a few operators will still mean quite a few are either using ”wrong” symbols or has to be represented as functions.

As an example, consider the dot and cross product of two vectors. The dot product might use ’*’ but what of the cross product? Unless we’re adding crazy stuff like unicode operators (which swift already demonstrated is a horrible idea)

I don’t have a perfect solution for this, but if we look at Fortran it already has specialized ways to perform operations across slices of a matrix, eg A(1:7, 4:9) = A(2:8, 4:9) + B(2:8, 2:7) which essentially means that for each element i the range col 1-7, row 4-9 we perform A[col, row] = A[col + 1, row] + B[col + 1, row - 2]

This is not to suggest Zig copy Fortran, but to show what it would be up against if trying to compete in that domain.

[isaachier]

@lerno the lucky thing with Fortran matrices is that those operations actually correspond to compiler intrinsics. They can be represented as vectors (https://llvm.org/docs/LangRef.html#vector-type), which have special SIMD support (#903). In those cases I strongly recommend the compiler support the type as a fundamental.

[ghost]

I was just reading in the docs about comptime and saw the max(...) example
https://ziglang.org/documentation/master/#Introducing-the-Compile-Time-Concept

fn max(comptime T: type, a: T, b: T) T {
    return if (a > b) a else b;
}

this should be a generic function but without overloading > it seems impossible to have generic functions for builtin ops because you have to duplicate them for your own types so functions can't be really generic in all cases?

functions can't be really generic in all cases

so this relates to anything that uses + - etc. as well which is why I'm bringing this up here although it would fit here as well

[BarabasGitHub]

Would it make sense to support the normal operators for arrays (and slices?) of fundamental types?
Fortran has that for all it's build in operations. See for example: https://en.wikipedia.org/wiki/Fortran_95_language_features#Arrays_2

I'm not sure if that's something Zig wants (only on basic operations +, -,*,/ imo). At least for fixed size arrays that would be nice. Then you don't need a big chunk of the Vector3D library stuff.

[mantielero]

Despite it is chinese to me, I found this thread pretty interesting. It is about what makes Fortran so good for numerical computation, and why it continues to be used instead of C, C++, Rust, ...

They say things like:

Multi-dimensional arrays, non-aliasing arrays and pointers, multi-dimensional array intrinsics, intrinsic array binary operators... all come with Fortran right out of the box

Rust has great potential to challenge Fortran in this domain, but numerics is mostly an afterthought in Rust.

So maybe, even if it is not implemented anytime soon, it looks like it might be a good idea to bear in mind this sort of things in mind.

[jeremyong]

I'd like to chime in and say that I think allowing users to define numeric types (not just complex, quaternion, but also dual units, dual quaternions, bivectors, etc) would be hugely beneficial (science, math, computer graphics).

Operator overloading can certainly be abused, but for numeric types, they really are the most (and some may say only) way to realistic express more complicated expressions, They obey laws of associativity, commutivity, and precedence as you would expect, and can be self-documenting used judiciously.

I'm looking at porting over some code from C++ now, and I have to say, the operator part is the only thing that makes me nervous (compare even a * b * ~a vs product(a, product(b, reverse(a)))). Expressions much longer than this (I have many) will veer on the unreadable side unfortunately.

[emekoi]

related #427

[momumi]

For math heavy code, it would be really nice to have some form of operator overloading. I'd really rather write z + z*y + y*y + 1 instead of z.add((y.mul(y)).add(z.mul(y))).add(1).

Instead of adding full blown operator overloading, how about introducing a new type say the MathType. The MathType would essential be identical to a struct but certain methods could be used to overload the basic operators.

Also, similar to how var keyword matches any type, the MathType would match the primitive numeric types as well as other MathType. Here's an example of how it might look:

const Complex = MathType {
    pub real: i64 = 0,
    pub imag: i64 = 0,

    pub fn add_left(self: Complex, rhs: MathType) Complex {
        // Complex + rhs
    }

    pub fn add_right(lhs: MathType, self: Complex) Complex {
        // lhs + Complex
        //
        // if lhs is a MathType but doesn't provide lhs.add_left, use this func
    }

    pub fn mul_left(self: Complex, rhs: MathType) Complex {
        // Complex * rhs
    }

    pub fn mul_right(lhs: MathType, self: Complex) Complex {
        // lhs * Complex
    }

    pub fn equal(self: Complex, other: MathType) bool {
        // lhs == Complex
        // Complex == lhs
    }

    pub fn assign(self: Complex, rhs: MathType) void {
        // Complex = rhs
    }
};

test "complex" {
    var z: Complex = 1;
    var y: Complex = Complex {.real = 1, .imag = 1};

    // calls all the right functions based on standard operator precedence
    z = z + z*y + y*y + 1; // z = z.add_left(y.mul_left(y).add_left(z.mul_left(y))).add_left(1)
    z = 1 + z; // z = z.add_right(1)
    assert(z == Complex{.real=3, .imag=1});
}

The MathType would help prevent operator overloading abuse since overloading would be restricted to only the MathType, and I doubt people would replace all their structs with MathType to abuse it either. For example this would prevent code like this:

var cout = Outstream.init();
cout << "hello world" << .Endl; // not possible: no way to overload << for []const u8 type

This would make the functions in std.math nicer too imo, since they could use function signatures like fn max(a:MathType, b:MathType) MathType instead of fn max(a:var, b:var) var.

I think the biggest issue is error handling. In most cases I think it is fine to require the use to validate input and just panic on bad values, e.g:

if (scale != 0) {
    var result = vector / scale;
}

However types like BigInt which might allocate memory to grow in size would probably need some way to fail while returning an error. For this case, maybe it could work like this:

const BigInt = MathType {
    // ...
    pub fn init(alloc: Allocator, MathType) !BigInt {
        //
    }

    pub fn add_left(self: !BigInt, rhs: MathType) !BigInt {
        if (self) |val| {
            // do calculation
            if (no_memory) {
                return error.OutOfMemory;
            }
            return new_val;
        } else |err| {
            // if we got an error forward it on to the caller
            return err;
        }
    }

    pub fn equal(self: BigInt, rhs: MathType) bool {
        // check equal
    }
}

test "bigint" {
    var a = BigInt.init(allocator, 1);
    var b = BigInt.init(allocator, 1);
    defer a.free();
    defer b.free();

    // if one of these math functions fails, it will propagate the error all
    // the way back to `a`
    var c = a*a + 2*a*b + b*b;
    defer c.free();

    // fails to compile, didn't unwrap the error
    // assert(c == 4);

    if (c) |val| {
        assert(val == 4);
    } else |err| {
        assert(false); // got some error
    }
}

This last example seems like it has issues since this expression var c = a*a + 2*a*b + b*b; would need to allocate memory for c. How do intermediate values get handled? Does the BigInt class need a destructor??

Anyway just throwing some ideas out there; I'm not fully sure on all this myself. Things look fine for objects with a static memory requirements, but objects with dynamic memory requirements seem a bit hairy.

[tauoverpi]

Could be possible to constrain overloading to anything without function calls which makes what you can do at the mercy of base operators. That at least forces a complete definition of the overload in one place and makes it painful enough to not go crazy on abuse. It's then not possible to allocate, concat, call out to magic procedures, etc, yet possible to gain convenience in operator use for at least some of the usecases while not violating the hidden control-flow rule.

[ityonemo]

for some reason I was thinking about this today. There is something clean about zig not allowing operator overloading for basic operators (each one implies no hidden function calls). But I think the issue is really "maybe we should have some functions that can have infix calls). I'd suggest, say allowing something like "a <+> b" for a generalized addition that you must import into the local scope by "usingnamespace" or something or other. It might be nice if you could also have unicode operators, like "a <⊕> b". You could also require such a thing to have parenthesis #114

Only problem is that this is not necessarily composable, if you have say, a matrix direct sum, and wanted to overload the internal +/- operators for this function (like let's say you want to do matrices over quaternions or GF256)... But I feel like there might be a clever way of doing this with comptime, even.

[andrewrk]

Proposal is too vague; closing.