diff --git a/README.md b/README.md index 4fa8af8a..aa6af1e6 100644 --- a/README.md +++ b/README.md @@ -1,135 +1,184 @@ # Regorus -**Regorus** is +**Regorus** is - *Rego*-*Rus(t)* - A fast, light-weight [Rego](https://www.openpolicyagent.org/docs/latest/policy-language/) interpreter written in Rust. - *Rigorous* - A rigorous enforcer of well-defined Rego semantics. Regorus is available as a library that can be easily integrated into your Rust projects. +```rust +use anyhow::Result; +use regorus::*; +use serde_json; + +fn main() -> Result<()> { + // Create an engine for evaluating Rego policies. + let mut engine = Engine::new(); + + // Add policy to the engine. + engine.add_policy( + // Filename to be associated with the policy. + "hello.rego".to_string(), + + // Rego policy that just sets a message. + r#" + package test + message = "Hello, World!" + "#.to_string() + )?; + + // Evaluate the policy, fetch the message and print it. + let results = engine.eval_query("data.test.message".to_string(), false)?; + println!("{}", serde_json::to_string_pretty(&results)?); + + Ok(()) +} +``` Regorus passes the [OPA v0.60.0 test-suite](https://www.openpolicyagent.org/docs/latest/ir/#test-suite) barring a few builtins. -See [OPA Conformance][#opa-conformance] below. +See [OPA Conformance](#opa-conformance) below. ## Getting Started -[examples/regorus](examples/regorus.rs) is an example program that shows how to integrate Regorus into your project and evaluate Rego policies. +[examples/regorus](https://github.com/microsoft/regorus/blob/main/examples/regorus.rs) is an example program that shows how to integrate Regorus into your project and evaluate Rego policies. To build and install it, do - cargo install --example regorus --path . - +```bash +$ cargo install --example regorus --path . +``` Check that the regorus example program is working - $ regorus - Usage: regorus - - Commands: - eval Evaluate a Rego Query - lex Tokenize a Rego policy - parse Parse a Rego policy - help Print this message or the help of the given subcommand(s) - - Options: - -h, --help Print help - -V, --version Print versionUsage: regorus +```bash +$ regorus +Usage: regorus +Commands: + eval Evaluate a Rego Query + lex Tokenize a Rego policy + parse Parse a Rego policy + help Print this message or the help of the given subcommand(s) + +Options: + -h, --help Print help + -V, --version Print version +``` First, let's evaluate a simple Rego expression `1*2+3` - regorus eval "1*2+3" +```bash +$ regorus eval "1*2+3" +``` This produces the following output +```json +{ + "result": [ { - "result": [ + "expressions": [ { - "expressions": [ - { - "value": 5, - "text": "1*2+3", - "location": { - "row": 1, - "col": 1 - } + "value": 5, + "text": "1*2+3", + "location": { + "row": 1, + "col": 1 } - ] } ] } + ] +} +``` Next, evaluate a sample [policy](examples/example.rego) and [input](examples/input.json) (borrowed from [Rego tutorial](https://www.openpolicyagent.org/docs/latest/#2-try-opa-eval)): - regorus eval -d examples/example.rego -i examples/input.json data.example +```bash +$ regorus eval -d examples/example.rego -i examples/input.json data.example +``` Finally, evaluate real-world [policies](tests/aci/) used in Azure Container Instances (ACI) - regorus eval -b tests/aci -d tests/aci/data.json -i tests/aci/input.json data.policy.mount_overlay=x +```bash +$ regorus eval -b tests/aci -d tests/aci/data.json -i tests/aci/input.json data.policy.mount_overlay=x +``` ## ACI Policies Regorus successfully passes the ACI policy test-suite. It is fast and can run each of the tests in a few milliseconds. - $ cargo test -r --test aci - Finished release [optimized + debuginfo] target(s) in 0.05s - Running tests/aci/main.rs (target/release/deps/aci-2cd8d21a893a2450) - aci/mount_device passed 3.863292ms - aci/mount_overlay passed 3.6905ms - aci/scratch_mount passed 3.643041ms - aci/create_container passed 5.046333ms - aci/shutdown_container passed 3.632ms - aci/scratch_unmount passed 3.631333ms - aci/unmount_overlay passed 3.609916ms - aci/unmount_device passed 3.626875ms - aci/load_fragment passed 4.045167ms +```bash +$ cargo test -r --test aci + Finished release [optimized + debuginfo] target(s) in 0.05s + Running tests/aci/main.rs (target/release/deps/aci-2cd8d21a893a2450) +aci/mount_device passed 3.863292ms +aci/mount_overlay passed 3.6905ms +aci/scratch_mount passed 3.643041ms +aci/create_container passed 5.046333ms +aci/shutdown_container passed 3.632ms +aci/scratch_unmount passed 3.631333ms +aci/unmount_overlay passed 3.609916ms +aci/unmount_device passed 3.626875ms +aci/load_fragment passed 4.045167ms +``` Run the ACI policies in the `tests/aci` directory, using data `tests/aci/data.json` and input `tests/aci/input.json`: - regorus eval -b tests/aci -d tests/aci/data.json -i tests/aci/input.json data.policy.mount_overlay=x - +```bash +$ regorus eval -b tests/aci -d tests/aci/data.json -i tests/aci/input.json data.policy.mount_overlay=x +``` Verify that [OPA](https://github.com/open-policy-agent/opa/releases) produces the same output - diff <(regorus eval -b tests/aci -d tests/aci/data.json -i tests/aci/input.json data.framework.mount_overlay=x) \ - <(opa eval -b tests/aci -d tests/aci/data.json -i tests/aci/input.json data.framework.mount_overlay=x) +```bash +$ diff <(regorus eval -b tests/aci -d tests/aci/data.json -i tests/aci/input.json data.framework.mount_overlay=x) \ + <(opa eval -b tests/aci -d tests/aci/data.json -i tests/aci/input.json data.framework.mount_overlay=x) +``` + ## Performance To check how fast Regorus runs on your system, first install a tool like [hyperfine](https://github.com/sharkdp/hyperfine). - cargo install hyperfine +```bash +$ cargo install hyperfine +``` Then benchmark evaluation of the ACI policies, - $ hyperfine "regorus eval -b tests/aci -d tests/aci/data.json -i tests/aci/input.json data.framework.mount_overlay=x" - Benchmark 1: regorus eval -b tests/aci -d tests/aci/data.json -i tests/aci/input.json data.framework.mount_overlay=x - Time (mean ± σ): 4.6 ms ± 0.2 ms [User: 4.1 ms, System: 0.4 ms] - Range (min … max): 4.4 ms … 6.0 ms 422 runs - -Compare it with OPA +```bash +$ hyperfine "regorus eval -b tests/aci -d tests/aci/data.json -i tests/aci/input.json data.framework.mount_overlay=x" +Benchmark 1: regorus eval -b tests/aci -d tests/aci/data.json -i tests/aci/input.json data.framework.mount_overlay=x + Time (mean ± σ): 4.6 ms ± 0.2 ms [User: 4.1 ms, System: 0.4 ms] + Range (min … max): 4.4 ms … 6.0 ms 422 runs +``` - $ hyperfine "opa eval -b tests/aci -d tests/aci/data.json -i tests/aci/input.json data.framework.mount_overlay=x" - Benchmark 1: opa eval -b tests/aci -d tests/aci/data.json -i tests/aci/input.json data.framework.mount_overlay=x - Time (mean ± σ): 45.2 ms ± 0.6 ms [User: 68.8 ms, System: 5.1 ms] - Range (min … max): 43.8 ms … 46.7 ms 62 runs +Compare it with OPA +```bash +$ hyperfine "opa eval -b tests/aci -d tests/aci/data.json -i tests/aci/input.json data.framework.mount_overlay=x" +Benchmark 1: opa eval -b tests/aci -d tests/aci/data.json -i tests/aci/input.json data.framework.mount_overlay=x + Time (mean ± σ): 45.2 ms ± 0.6 ms [User: 68.8 ms, System: 5.1 ms] + Range (min … max): 43.8 ms … 46.7 ms 62 runs +``` ## OPA Conformance -Regorus has been verified to be compliant with [OPA v0.60.0](https://github.com/open-policy-agent/opa/releases/tag/v0.60.0) -using a [test driver](tests/opa.rs) that loads and runs the OPA testsuite using Regorus, and verifies that expected outputs +Regorus has been verified to be compliant with [OPA v0.60.0](https://github.com/open-policy-agent/opa/releases/tag/v0.60.0) +using a [test driver](https://github.com/microsoft/regorus/blob/main/tests/opa.rs) that loads and runs the OPA testsuite using Regorus, and verifies that expected outputs are produced. The test driver can be invoked by running: -``` -cargo test -r --test opa +```bash +$ cargo test -r --test opa ``` -Currently, Regorus passes all the non-builtin specific tests. See [passing tests suites](tests/opa.passing). +Currently, Regorus passes all the non-builtin specific tests. See [passing tests suites](https://github.com/microsoft/regorus/blob/main/tests/opa.passing). The following test suites don't pass fully due to mising builtins: - `cryptoparsersaprivatekeys` @@ -167,15 +216,16 @@ The following test suites don't pass fully due to mising builtins: They are captured in the following [github issues](https://github.com/microsoft/regorus/issues?q=is%3Aopen+is%3Aissue+label%3Alib). + ### Grammar -The grammar used by Regorus to parse Rego policies is described in [grammar.md](docs/grammar.md) in both EBNF and RailRoad Diagram formats. +The grammar used by Regorus to parse Rego policies is described in [grammar.md](https://github.com/microsoft/regorus/blob/main/docs/grammar.md) in both [W3C EBNF](https://www.w3.org/Notation.html) and [RailRoad Diagram](https://en.wikipedia.org/wiki/Syntax_diagram) formats. ## Contributing This project welcomes contributions and suggestions. Most contributions require you to agree to a Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us -the rights to use your contribution. For details, visit https://cla.opensource.microsoft.com. +the rights to use your contribution. For details, visit . When you submit a pull request, a CLA bot will automatically determine whether you need to provide a CLA and decorate the PR appropriately (e.g., status check, comment). Simply follow the instructions @@ -187,8 +237,8 @@ contact [opencode@microsoft.com](mailto:opencode@microsoft.com) with any additio ## Trademarks -This project may contain trademarks or logos for projects, products, or services. Authorized use of Microsoft -trademarks or logos is subject to and must follow +This project may contain trademarks or logos for projects, products, or services. Authorized use of Microsoft +trademarks or logos is subject to and must follow [Microsoft's Trademark & Brand Guidelines](https://www.microsoft.com/en-us/legal/intellectualproperty/trademarks/usage/general). Use of Microsoft trademarks or logos in modified versions of this project must not cause confusion or imply Microsoft sponsorship. Any use of third-party trademarks or logos are subject to those third-party's policies. diff --git a/examples/regorus.rs b/examples/regorus.rs index 89f6ee4f..16c13a07 100644 --- a/examples/regorus.rs +++ b/examples/regorus.rs @@ -2,7 +2,6 @@ // Licensed under the MIT License. use anyhow::{bail, Result}; -use clap::{Parser, Subcommand}; fn rego_eval( bundles: &[String], @@ -75,16 +74,18 @@ fn rego_eval( } fn rego_lex(file: String, verbose: bool) -> Result<()> { + use regorus::unstable::*; + // Create source. - let source = regorus::Source::from_file(file)?; + let source = Source::from_file(file)?; // Create lexer. - let mut lexer = regorus::Lexer::new(&source); + let mut lexer = Lexer::new(&source); // Read tokens until EOF. loop { let token = lexer.next_token()?; - if token.0 == regorus::TokenKind::Eof { + if token.0 == TokenKind::Eof { break; } @@ -100,18 +101,20 @@ fn rego_lex(file: String, verbose: bool) -> Result<()> { } fn rego_parse(file: String) -> Result<()> { + use regorus::unstable::*; + // Create source. - let source = regorus::Source::from_file(file)?; + let source = Source::from_file(file)?; // Create a parser and parse the source. - let mut parser = regorus::Parser::new(&source)?; + let mut parser = Parser::new(&source)?; let ast = parser.parse()?; println!("{ast:#?}"); Ok(()) } -#[derive(Subcommand)] +#[derive(clap::Subcommand)] enum RegorusCommand { /// Evaluate a Rego Query. Eval { @@ -164,6 +167,7 @@ struct Cli { } fn main() -> Result<()> { + use clap::Parser; env_logger::builder() .format_level(false) .format_timestamp(None) diff --git a/src/interpreter.rs b/src/interpreter.rs index d7697edc..556e9357 100644 --- a/src/interpreter.rs +++ b/src/interpreter.rs @@ -226,10 +226,6 @@ impl Interpreter { self.modules = modules.to_vec(); } - pub fn get_modules_mut(&mut self) -> &mut Vec> { - &mut self.modules - } - pub fn set_init_data(&mut self, init_data: Value) { self.init_data = init_data; } diff --git a/src/lib.rs b/src/lib.rs index a9b0404b..b6bc2b36 100644 --- a/src/lib.rs +++ b/src/lib.rs @@ -1,22 +1,30 @@ // Copyright (c) Microsoft Corporation. // Licensed under the MIT License. -pub mod ast; +// Use README.md as crate documentation. +#![doc = include_str!(concat!(env!("CARGO_MANIFEST_DIR"), "/README.md"))] + +mod ast; mod builtins; -pub mod engine; -pub mod interpreter; -pub mod lexer; -pub mod number; -pub mod parser; -pub mod scheduler; +mod engine; +mod interpreter; +mod lexer; +mod number; +mod parser; +mod scheduler; mod utils; -pub mod value; +mod value; + +pub use engine::Engine; +pub use interpreter::{QueryResult, QueryResults}; +pub use value::Value; + +/// Items in `unstable` are likely to change. +pub mod unstable { + pub use crate::ast::*; + pub use crate::lexer::*; + pub use crate::parser::*; +} -pub use ast::*; -pub use engine::*; -pub use interpreter::*; -pub use lexer::*; -pub use number::*; -pub use parser::*; -pub use scheduler::*; -pub use value::*; +#[cfg(test)] +mod tests; diff --git a/src/number.rs b/src/number.rs index 237e72ff..385854c3 100644 --- a/src/number.rs +++ b/src/number.rs @@ -473,3 +473,14 @@ impl Number { .unwrap_or("".to_string()) } } + +#[cfg(test)] +mod test { + use crate::number::*; + + #[test] + fn display_number() { + let n = Number::from(123456f64); + assert_eq!(format!("{}", n.format_decimal()), "123456"); + } +} diff --git a/src/scheduler.rs b/src/scheduler.rs index 9f677bc2..d5ce0697 100644 --- a/src/scheduler.rs +++ b/src/scheduler.rs @@ -36,6 +36,7 @@ pub enum SortResult { // The order in which statements must be executed. Order(Vec), // List of statements comprising a cycle for a given var. + #[allow(unused)] Cycle(String, Vec), } diff --git a/tests/interpreter/mod.rs b/src/tests/interpreter/mod.rs similarity index 99% rename from tests/interpreter/mod.rs rename to src/tests/interpreter/mod.rs index 61310c52..3a6decd3 100644 --- a/tests/interpreter/mod.rs +++ b/src/tests/interpreter/mod.rs @@ -3,8 +3,9 @@ use std::env; +use crate::*; + use anyhow::{bail, Result}; -use regorus::*; use serde::{ser::SerializeMap, Deserialize, Deserializer, Serialize, Serializer}; use test_generator::test_resources; @@ -140,7 +141,7 @@ pub fn eval_file( enable_tracing: bool, strict: bool, ) -> Result> { - let mut engine: Engine = engine::Engine::new(); + let mut engine: Engine = Engine::new(); engine.set_strict_builtin_errors(strict); let mut results = vec![]; diff --git a/src/tests/mod.rs b/src/tests/mod.rs new file mode 100644 index 00000000..4adb12f6 --- /dev/null +++ b/src/tests/mod.rs @@ -0,0 +1,5 @@ +// Copyright (c) Microsoft Corporation. +// Licensed under the MIT License. + +mod interpreter; +mod scheduler; diff --git a/tests/scheduler/analyzer/mod.rs b/src/tests/scheduler/analyzer/mod.rs similarity index 95% rename from tests/scheduler/analyzer/mod.rs rename to src/tests/scheduler/analyzer/mod.rs index 0be7e57f..1409015e 100644 --- a/tests/scheduler/analyzer/mod.rs +++ b/src/tests/scheduler/analyzer/mod.rs @@ -1,9 +1,8 @@ // Copyright (c) Microsoft Corporation. // Licensed under the MIT License. +use crate::{ast::*, lexer::*, parser::*, scheduler::*}; use anyhow::{bail, Result}; -use regorus::scheduler::*; -use regorus::*; use serde::{Deserialize, Serialize}; use test_generator::test_resources; @@ -49,7 +48,7 @@ fn analyze_file(regos: &[String], expected_scopes: &[Scope]) -> Result<()> { let analyzer = Analyzer::new(); let schedule = analyzer.analyze(&modules)?; - let mut scopes: Vec<(Ref, ®orus::Scope)> = schedule + let mut scopes: Vec<(Ref, &crate::scheduler::Scope)> = schedule .scopes .iter() .map(|(r, s)| (r.clone(), s)) diff --git a/tests/scheduler/mod.rs b/src/tests/scheduler/mod.rs similarity index 99% rename from tests/scheduler/mod.rs rename to src/tests/scheduler/mod.rs index a8d3e072..9462dc85 100644 --- a/tests/scheduler/mod.rs +++ b/src/tests/scheduler/mod.rs @@ -1,10 +1,9 @@ // Copyright (c) Microsoft Corporation. // Licensed under the MIT License. +use crate::scheduler::*; use anyhow::{bail, Result}; -use regorus::scheduler::*; - mod analyzer; fn make_info(definitions: &[(&'static str, &[&'static str])]) -> StmtInfo<&'static str> { diff --git a/tests/lexer/mod.rs b/tests/lexer/mod.rs index 3c3b7eef..7d6d7ac1 100644 --- a/tests/lexer/mod.rs +++ b/tests/lexer/mod.rs @@ -2,7 +2,7 @@ // Licensed under the MIT License. use anyhow::{bail, Result}; -use regorus::*; +use regorus::unstable::*; use serde::{Deserialize, Serialize}; use test_generator::test_resources; diff --git a/tests/tests.rs b/tests/mod.rs similarity index 77% rename from tests/tests.rs rename to tests/mod.rs index c735c9a8..aa08f980 100644 --- a/tests/tests.rs +++ b/tests/mod.rs @@ -1,8 +1,6 @@ // Copyright (c) Microsoft Corporation. // Licensed under the MIT License. -mod interpreter; mod lexer; mod parser; -mod scheduler; mod value; diff --git a/tests/parser/mod.rs b/tests/parser/mod.rs index 7ffea6fd..65d721ff 100644 --- a/tests/parser/mod.rs +++ b/tests/parser/mod.rs @@ -2,7 +2,7 @@ // Licensed under the MIT License. use anyhow::{anyhow, bail, Result}; -use regorus::*; +use regorus::{unstable::*, *}; use serde::{Deserialize, Serialize}; use test_generator::test_resources; diff --git a/tests/value/mod.rs b/tests/value/mod.rs index f252bfa8..7e449903 100644 --- a/tests/value/mod.rs +++ b/tests/value/mod.rs @@ -66,12 +66,6 @@ fn serialize_number() -> Result<()> { Ok(()) } -#[test] -fn display_number() { - let n = Number::from(123456f64); - assert_eq!(format!("{}", n.format_decimal()), "123456"); -} - #[test] fn serialize_string() -> Result<()> { assert_eq!(