From f83183070449c79f2dd39082e32f93bea701e133 Mon Sep 17 00:00:00 2001 From: Dmitry Balashov Date: Mon, 11 Sep 2023 12:24:07 +0700 Subject: [PATCH 1/3] docs: migrate document from hyperledger/iroha#3795 Signed-off-by: Dmitry Balashov --- 000-config-overhaul.md | 912 +++++++++++++++++++++++++++++++++++++++++ 1 file changed, 912 insertions(+) create mode 100644 000-config-overhaul.md diff --git a/000-config-overhaul.md b/000-config-overhaul.md new file mode 100644 index 0000000..b7376f1 --- /dev/null +++ b/000-config-overhaul.md @@ -0,0 +1,912 @@ +# Iroha 2 Configuration Overhaul RFC + +## Introduction + +Configuration is often the first point of interaction for users with Hyperledger Iroha 2. It's crucial to make this +experience as smooth as possible. However, the current system, marred by issues like redundant fields, ambiguous naming, +and unclear error messages, necessitates a thorough review. This RFC sets out to propose vital improvements for a more +intuitive and efficient configuration experience. + +## Background + +### Redundant Fields + +There are configuration fields that shouldn't be initialized by the user, meaning that including them in the +configuration reference is redundant. For example, here's the +[excerpt](https://github.com/hyperledger/iroha/blob/35ba182e1d1b6b594712cf63bf448a2edefcf2cd/docs/source/references/config.md#sumeragi-default-null-values) +about configuration options for Sumeragi: + +> **Sumeragi: default `null` values** +> +> A special note about sumeragi fields with `null` as default: only the `trusted_peers` field out of the three can be +> initialized via a provided file or an environment variable. +> +> The other two fields, namely `key_pair` and `peer_id`, go through a process of finalization where their values are +> derived from the corresponding ones in the uppermost Iroha config (using its `public_key` and `private_key` fields) or +> the Torii config (via its `p2p_addr`). This ensures that these linked fields stay in sync, and prevents the programmer +> error when different values are provided to these field pairs. Providing either `sumeragi.key_pair` or +> `sumeragi.peer_id` by hand will result in an error, as it should never be done directly. + +As we can see, if the user tries to initialize `key_pair` and `peer_id` via the config file or env variables, they'll +get an error. We are creating a terrible user experience by presenting the user with an option to configure something +they should not be configuring. + +We should not expose such fields to the user as they are implementation details. + +### Naming Issues + +The configuration parameter naming convention poses the following challenges. + +#### Overcomplicated ENV Variables + +ENV variables for all configuration parameters are named according to the internal module names. These technical +designations might resonate with developers but can be confusing or alienating for users not deeply immersed in the +project's inner mechanisms. The +[POSIX specification](https://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap08.html#tag_08) actively +encourages environment variables with succinct names. Given that a number of standard environment variable names exist, +and their value can be redefined on a per-process basis, it is recommended to use generic environment variables as much +as possible, before using specific names prefixed with `IROHA_`. + +#### Inconsistent Naming of ENV Variables + +The output from the `iroha --help` command displays a mix of naming conventions: + +``` +Iroha 2 is configured via environment variables: + IROHA2_CONFIG_PATH is the location of your `config.json` or `config.json5` + IROHA2_GENESIS_PATH is the location of your `genesis.json` or `genesis.json5` +... + IROHA_TORII: Torii (gateway) endpoint configuration + IROHA_SUMERAGI: Sumeragi (emperor) consensus configuration + IROHA_KURA: Kura (storage). Configuration of block storage + IROHA_BLOCK_SYNC: Block synchronisation configuration +... +``` + +The inconsistency between the `IROHA2` and `IROHA` prefixes can be confusing. Of the two, the `IROHA` prefix is +preferred, as users are discouraged from using Iroha 1 in any new projects. + +#### Inconsistent Naming of Configuration Parameters + +Current configuration demonstrates some inconsistencies in parameter naming, leading to potential user confusion. To +address this, some refinements are proposed: + +- The `torii` section uses terms like `p2p_addr`, `api_url`, and `telemetry_url`. These are actually addresses in the + form of `localhost:8080` and not full URLs, which would include components like protocol and path. To standardize, + these should all be changed to use `addr` (e.g., `addr_p2p`, `addr_api`, `addr_telemetry`). +- Consider moving addresses to their respective domain scopes. For example, instead of `torii.telemetry_url`, we could + have `telemetry.addr`. +- In the `logger` section, the term `max_log_level` could be simplified to `log_level` or just `level` to make it more + concise without losing meaning. Although users are more likely to set this parameter using an environment variable. +- In the `wsv` section, the field `wasm_runtime_config` has a redundant `_config` suffix. Given these are all + _configuration_ parameters, such suffixes can be eliminated for clarity. + +Implementing a consistent naming convention like this will simplify the user's configuration process and reduce +ambiguity. + +#### Redundant SCREAMING_CASE in JSON Configuration + +The adoption of SCREAMING_CASE for parameters within the JSON configuration appears out of place. Such uppercase field +names might be customary for environment variables, but in a JSON setting, they don't offer any distinctive benefit. +Here's a snippet from the current configuration showcasing this: + +```json +{ + "PUBLIC_KEY": null, + "PRIVATE_KEY": null, + "DISABLE_PANIC_TERMINAL_COLORS": false +} +``` + +--- + +In summary, these issues underscore the need for clearer, more intuitive naming conventions in Iroha 2's configuration. + +### Rustisms in the Configuration Reference + +Configuration reference contains Rust-specific expressions that wouldn't make sense to the end-user who is configuring +Iroha and might not be familiar with Rust. Here are some examples: + +- [Using `Option<..>`](https://github.com/hyperledger/iroha/blob/35ba182e1d1b6b594712cf63bf448a2edefcf2cd/docs/source/references/config.md#option) +- [Using `std::path::PathBuf`](https://github.com/hyperledger/iroha/blob/35ba182e1d1b6b594712cf63bf448a2edefcf2cd/docs/source/references/config.md#loggerlog_file_path) + +The use of rustisms like that overcomplicates the configuration reference and affects the user experience. Further, it +doesn't actually provide any extra information to a user familiar with Rust, as environment variables and JSON values +cannot be `None` due to being stringly typed and null-able regardless of whether the type specifies `Option`. + +### Unhelpful Error Messages + +Some of the error messages related to Iroha configuration are inconsistent and misleading. Here's a quick rundown of +various scenarios, associated errors and possible confusions. + +#### 1. No configuration file + +Running Iroha without a configuration file results in the following log and error: + +``` +IROHA_TORII: environment variable not found +IROHA_SUMERAGI: environment variable not found +IROHA_KURA: environment variable not found +IROHA_BLOCK_SYNC: environment variable not found +IROHA_PUBLIC_KEY: environment variable not found +IROHA_PRIVATE_KEY: environment variable not found +IROHA_GENESIS: environment variable not found +Configuration file not found. Using environment variables as fallback. +Error: + 0: Please add `PUBLIC_KEY and PRIVATE_KEY` to the configuration. +``` + +Several things are wrong with this error message: + +- The information about using environment variables as fallback comes after the messages about them not being found. +- Hinting to add `PUBLIC_KEY` and `PRIVATE_KEY` to the configuration does not explain what happened (the absence of the + config file and no environment variables to fall back to). This hint is also not helpful as there is no information on + whether public and private keys should be added to the config file or to ENV variables, and no information about the + ENV variables these fields are mapped to. + +#### 2. Path to non-existent config file + +Providing a path to a configuration file that doesn't exist results in the exact same error as if there was no +configuration file specified at all. + +When a user specifies a path to a configuration file, it is generally with the expectation that the file will be read +and its settings applied. If the file doesn't exist, an immediate and clear error message should be thrown to alert the +user of this issue. + +#### 3. Empty config file + +Running Iroha with an empty `config.json` results in the error that does not provide any information about the fallback +to the environment variables even though it does happen. This behaviour is inconsistent. + +``` +Error: + 0: Please add `PUBLIC_KEY and PRIVATE_KEY` to the configuration. +``` + +#### 4. Config file with only `PUBLIC_KEY` and `PRIVATE_KEY` specified + +Running Iroha with a config file that only contains `PUBLIC_KEY` and `PRIVATE_KEY` results in the following error: + +``` +Error: + 0: Please add `p2p_addr` to the configuration. +``` + +This error message is misleading and inconsistent: + +- If the `p2p_addr` field is added to the config file, the error stays the same. +- The `p2p_addr` field is not a root-level field, it's actually a part of the `TORII` configuration and should be + `TORII.P2P_ADDR`. +- Error messages asking to add public and private keys use uppercase for configuration fields, while in this error the + field name is written in lowercase. + +#### 5. Config file with only `PUBLIC_KEY`, `PRIVATE_KEY`, and `TORII.P2P_ADDR` specified + +Running Iroha with a config file that only contains `PUBLIC_KEY`, `PRIVATE_KEY`, and `TORII.P2P_ADDR` results in the +following error: + +``` +Error: + 0: Please add `api_url` to the configuration. +``` + +Comparing this scenario to the previous one, we can notice that the `TORII.API_URL` was missing before as well but the +previous error message didn't mention it. + +There are also two inconsistencies: + +- Similar to `p2p_addr` in the previous scenario, this field name is lowercase, while for public and private keys it was + uppercase. +- While `TORII.P2P_ADDR` and `TORII.API_URL` share the same root-level (`TORII`), their names are different: "addr" + derived from "address" and "URL". Logically these are both addresses or URLs, and it would make sense for the names to + be aligned. + +#### 6. Config file with invalid `PUBLIC_KEY` + +Providing an invalid `PUBLIC_KEY` in the `config.json` results in the same error as when there is no config file at all, +or an empty config file: + +``` +Error: + 0: Please add `PUBLIC_KEY and PRIVATE_KEY` to the configuration. +``` + +This is not a helpful error message as it does not mention what exactly is wrong, why and where the parsing failed. + +#### 7. Invalid `IROHA2_PUBLIC_KEY` environment variable + +Providing an invalid `IROHA2_PUBLIC_KEY` environment variable results in the following error: + +``` +Error: + 0: Failed to build configuration from env + 1: Failed to deserialize the field `IROHA_PUBLIC_KEY`: JSON5: Key could not be parsed. Key could not be parsed. Invalid character 's' at position 4 + 2: JSON5: Key could not be parsed. Key could not be parsed. Invalid character 's' at position 4 +``` + +While this message is more helpful than the ones we discussed above, there are still issues: + +- The message contains repetition. +- Without an input snippet, the `Invalid character 's' at position 4` part of the message is not as helpful as it could + be. + +#### 8. Config file with extra fields + +Providing a configuration file with extra fields that are not supposed to be there does not produce any errors at all. + +This might lead to a bad user experience when user expects some options to apply, but doesn't have any idea that those +in fact are silently ignored. + +#### 9. Specifying `SUMERAGI.PUBLIC_KEY` + +As you can see in the "Redundant fields" section, user should not specify any non-null value for `SUMERAGI.PUBLIC_KEY` +parameter. However, if they do, they will get the following error: + +``` +Error: + 0: Please add `PUBLIC_KEY and PRIVATE_KEY` to the configuration. +``` + +The problem is that the error message tells something completely unrelated to the actual cause. This produces a terrible +experience for users. + +#### 10. Config file is an invalid JSON file + +Providing an invalid JSON file as a config results in the same error we've seen before, which is not useful at all in +this particular case and doesn't tell the user anything about the invalid JSON: + +``` +Error: + 0: Please add `PUBLIC_KEY and PRIVATE_KEY` to the configuration. +``` + +### Ambiguity in Setting Root-Level Fields via ENV + +To understand the issue with ambiguity at the root-level fields, let's once again look at the excerpt from the +[configuration reference](https://github.com/hyperledger/iroha/blob/35ba182e1d1b6b594712cf63bf448a2edefcf2cd/docs/source/references/config.md#network): + +> **`network`** +> +> Network configuration +> +> Has type `Option`. Can be configured via environment variable `IROHA_NETWORK` +> +> ```json +> { +> "ACTOR_CHANNEL_CAPACITY": 100 +> } +> ``` + +The current configuration treats namespaces like `network` as both a grouping mechanism for specific fields (e.g., +`network.*`) and as independent parameters. This creates ambiguity: for instance, what would happen if an environment +variable like `IROHA_NETWORK` is set? Would it affect nested fields such as `IROHA_NETWORK_CAPACITY`? + +This dual role for namespaces is not just limited to `network`, but applies to all other namespaces in the +configuration. This approach is somewhat exotic and not well-documented, leading to potential confusion and +unpredictability. Given these challenges, it might be beneficial to simplify the configuration by treating namespaces +strictly as prefixes for other parameters, rather than as standalone configuration values. + +### Chaotic Code Organisation + +Internally, not all the configuration-related logic is contained within a single `iroha_config` crate. Instead, some of +the configuration resolution logic is located in other crates, such as `iroha_cli`. This makes the configuration-related +code error-prone and harder to maintain. + +## Proposals + +The proposals outlined in this section are designed to collectively enhance the configuration system within Iroha. +However, they are not mutually required. Each proposal can be considered and implemented independently. + +### Proposal 1 - Use TOML + +#### Objective + +Transition to TOML as Iroha's standard configuration format to provide a cleaner, more intuitive setup and to align with +Rust community standards. + +#### Rationale + +- **Human-Friendly Syntax:** TOML's format is easier to read and understand compared to JSON, especially given its + support for comments. +- **Rust Ecosystem Alignment:** Adopting TOML ensures that Iroha remains consistent with prevalent configuration + practices in the Rust community. +- **Elimination of SCREAMING_CASE:** By using TOML, Iroha can also abandon the use of SCREAMING_CASE in configuration, + making it more in line with conventional formatting standards. + +#### Summary + +Adopting TOML allows Iroha to embrace a more readable and conventional configuration format. The move away from JSON +will simplify user interaction and align Iroha better with the broader Rust ecosystem. + +#### See Also + +- [Proposal 3 - Consistent Naming of Configuration Parameters](#proposal-3---consistent-naming-of-configuration-parameters) +- [Proposal 4 - Better Aliases for ENV](#proposal-4---better-aliases-for-env) +- [Proposal 5 - Define Deprecation and Migration Policy](#proposal-5---define-deprecation-and-migration-policy) + +### Proposal 2 - Reference Before Implementation + +#### Objective + +Establish a clear and comprehensive configuration reference prior to actual code implementation. This approach ensures +that any new features or changes are well-documented, understandable, and in alignment with the project's goals. + +#### Rationale + +- **Clarity and Direction:** By laying out a detailed reference before diving into coding, we make sure everyone's on + the same page and knows the direction we're headed. +- **Efficient Development:** Prevents potential backtracking or revisions in the coding phase, as developers will be + working with a clear guide. +- **Enhanced Collaboration:** A preliminary reference can be critiqued, discussed, and iterated upon, leading to better + decisions in the design phase. +- **User Engagement:** Early availability of a configuration reference aids in early user feedback, ensuring that the + implementation is user-centric. + +#### Sub-points and References: + +- **[Proposal 1 - Use TOML](#proposal-1---use-toml)**: Settling on TOML as the standard configuration format before + diving into the coding phase will prevent format-switching complications. +- **[Proposal 3 - Consistent Naming of Configuration Parameters](#proposal-3---consistent-naming-of-configuration-parameters)**: + Prior to implementation, deciding on naming conventions and potential changes (like nesting keys under `iroha` or + changing parameter names) will streamline development and ensure consistency. +- **[Proposal 4 - Better Aliases for ENV](#proposal-4---better-aliases-for-env)**: Defining environment variable aliases + in advance allows for more predictable and user-friendly configuration handling. +- **[Proposal 5 - Define Deprecation and Migration Policy](#proposal-5---define-deprecation-and-migration-policy)**: + Establishing guidelines on how to handle deprecated features or migrations ensures smoother transitions and fewer + surprises for users. + +#### Summary + +Developing a configuration reference before the actual implementation can lead to a more coherent, user-friendly, and +efficient development process. By addressing key points beforehand, like naming conventions, format choices, and alias +definitions, Iroha 2 will be better positioned to deliver a configuration system that resonates with its users. + +#### See Also + +- [[suggestion] Remove the requirement to set `sumeragi.key_pair` and `sumeragi.peer_id` to `null` in the configuration · Issue #3504 · hyperledger/iroha](https://github.com/hyperledger/iroha/issues/3504) +- [[suggestion] Get rid of "rustisms" in the configuration reference · Issue #3505 · hyperledger/iroha](https://github.com/hyperledger/iroha/issues/3505) +- [[suggestion] Enhance configuration reference format · Issue #3507 · hyperledger/iroha](https://github.com/hyperledger/iroha/issues/3507) + +### Proposal 3 - Consistent Naming of Configuration Parameters + +#### Objective + +Standardise the naming conventions of Iroha's configuration parameters, enhancing their intuitiveness and consistency +for improved user interaction. + +#### Rationale + +- **Clarity:** Consistent naming reduces ambiguity, allowing users to more easily understand and predict parameter + names. +- **Ease of Documentation:** A unified naming convention simplifies the documentation process, ensuring coherence and + easier updates. +- **Error Prevention:** Predictable and clear naming reduces the risk of configuration errors by users. + +#### Proposed Name Changes + +Here are the proposed changes for more consistent naming: + +- `torii`: + - Rename from `p2p_addr` to `addr_p2p`. + - Rename from `api_url` to `addr_api`. + - Rename from `telemetry_url` to `addr_telemetry`. +- `logger`: + - Rename from `max_log_level` to `log_level` or simply `level`. +- `wsv`: + - Rename from `wasm_runtime_config` to `wasm_runtime`. + +For the sake of uniformity, it might be beneficial to nest the root `public_key` and `private_key` under an `iroha` +namespace, resulting in `iroha.public_key` and `iroha.private_key`[^1]. + +Please note that this list is not exhaustive. During the configuration reference design process, we might encounter +further areas of improvement and additional renaming suggestions, which shall be reflected as updates to the RFC. + +#### Summary + +This proposal aims to standardise the naming conventions within Iroha's configuration parameters, ensuring clarity and +predictability, which will enhance user experience and streamline documentation efforts. + +#### See Also + +- [Proposal 1 - Use TOML](#proposal-1---use-toml) +- [Proposal 4 - Better Aliases for ENV](#proposal-4---better-aliases-for-env) +- [Proposal 5 - Define Deprecation and Migration Policy](#proposal-5---define-deprecation-and-migration-policy) +- [Representation of key pair in configuration files · Issue #2135 · hyperledger/iroha](https://github.com/hyperledger/iroha/issues/2135) + +### Proposal 4 - Better Aliases for ENV + +#### Objective + +Introduce intuitive and standardized environment variable names, moving away from internal code-names, to simplify user +experience. + +#### Rationale + +- **User Accessibility:** Generifying names improves understanding and avoids confusion. Using internal module names can + be restrictive and lead to misconfiguration. +- **Consistency:** A uniform naming convention for environment variables enhances predictability, aiding users in + setting up configurations without constantly referring to documentation. +- **Transition Support:** New aliases will coexist with existing ones, ensuring no disruptions and aiding users during + the transitional phase. + +#### Proposed Changes + +1. **Rename Based on Functionality:** Shift from code-based naming to function-based naming. For example: + - `TORII_API_URL` becomes `API_ENDPOINT_URL`. + - Convert `KURA_INIT_MODE` to `BLOCK_REVALIDATION_MODE`. + - `LOGGER_MAX_LOG_LEVEL` becomes `MAX_LOG_LEVEL` or simply `LOG_LEVEL` +2. **Document All Aliases:** Clearly list main variables and their respective aliases in the configuration reference. +3. **Trace Resolution in Logs:** For clarity, ensure that when an alias is used, its resolution to the actual parameter + is evident in logs. + +#### Summary + +By transitioning to more user-friendly ENV variable names and reducing reliance on internal naming conventions, we aim +to make Iroha 2's configuration process more intuitive and error-free. + +#### See Also + +- [Proposal 2 - Reference Before Implementation](#proposal-2---reference-before-implementation) +- [Proposal 3 - Consistent Naming of Configuration Parameters](#proposal-3---consistent-naming-of-configuration-parameters) +- [Proposal 7 - Trace Configuration Resolution](#proposal-7---trace-configuration-resolution) + +### Proposal 5 - Define Deprecation and Migration Policy + +#### Objective + +To establish a clear and structured policy for marking features as deprecated and guiding users towards adopting newer +alternatives or configurations. + +#### Rationale + +- **User Trust:** A well-documented deprecation policy ensures that users can trust the software's lifecycle and + understand the trajectory of its development. +- **Smooth Transitions:** Clear policies guide users in migrating from older configurations or features to newer ones, + ensuring continuity of operations. +- **Maintainability:** Developers benefit from clear guidelines on when and how to retire older code, helping in + reducing technical debt. +- **Clear Communication:** A standardized policy ensures that all users, from developers to end-users, have a clear + understanding of changes. + +#### Proposed Policy + +1. **Notification:** When a feature or configuration is marked for deprecation, it should be communicated via release + notes, documentation updates, and, when relevant, through warnings within the software itself. +2. **Grace Period:** Provide users a sufficient grace period (e.g., a few versions ahead) before the deprecated feature + is removed. This period allows them to adjust and migrate without sudden disruptions. +3. **Migration Guidelines:** Offer detailed documentation on how to transition from the deprecated feature or + configuration to its modern replacement. +4. **Clear Timeline:** Specify a clear timeline from the moment of deprecation to the removal, giving users ample time + to prepare. +5. **Deprecated Feature Registry:** Maintain a list or registry of all deprecated features and configurations, along + with their slated removal dates, for users to reference. + +#### Summary + +Introducing a structured deprecation and migration policy not only establishes trust with users but also ensures the +software evolves efficiently, balancing innovation with stability. + +### Proposal 6 - Exhaustive Error Messages + +#### Objective + +To enhance user troubleshooting by offering comprehensive and descriptive error messages during the configuration stage, +specifically for two main types of configuration errors. + +#### Rationale + +Diagnosing and resolving configuration errors can be challenging without detailed insights. A more descriptive error +messaging system offers the following benefits: + +1. **Parsing with Precision:** + - **Highlight Invalid Data's Location:** Pinpoint the exact location of errors, making it easier for users to + address. + - **Identify Missing Fields:** Rapidly flag any necessary but absent data, speeding up the configuration process. + - **Spot Unknown Fields:** Alert users to any unrecognized fields, helping them to quickly catch and fix + typographical mistakes or redundant information. +2. **Clear Directions for Domain-Specific Errors:** + - **Highlighting Semantic Issues:** Inform users about errors or warnings tied to the higher-level domain details + specific to Iroha. This ensures that users are aware of any nuances or complexities related to Iroha itself, + allowing for a more refined and accurate configuration. +3. **Eliminate Guesswork:** + - **Full Transparency in Reporting:** Ensure users are immediately and fully informed of any issues, preventing + oversights and minimizing confusion. +4. **Consistency in Communication:** + - **Uniform, Informative Messaging:** Maintain a consistent style across error messages, bolstering clarity and user + understanding. +5. **Streamline Multi-Error Solutions:** + - **Bulk Reporting of Missing Fields:** If several fields are missing, display them all simultaneously for more + efficient troubleshooting. + +#### Potential Solutions & Samples + +To illustrate the potential of this system, a Proof-of-Concept (PoC) repository should be created, showcasing the +clarity and utility of these enhanced error messages in Rust. + +#### Summary + +Implementing exhaustive error messages will drastically enhance user troubleshooting, ensuring a smoother and more +efficient configuration process for Iroha users. + +#### See Also + +- [[suggestion] Enhance configuration parsing with precise field locations · Issue #3470 · hyperledger/iroha](https://github.com/hyperledger/iroha/issues/3470) + +### Proposal 7 - Trace Configuration Resolution + +#### Objective + +Implement a mechanism that displays the sequence in which configuration is determined. This allows users to grasp the +sources from which configuration values are derived, whether they're being overridden, and which resort to default +values. + +#### Rationale + +Transparency in configuration resolution is vital for troubleshooting and ensuring the correct setup. By providing clear +feedback on how parameters are set, users can identify potential misconfigurations or conflicts quickly. + +#### Proposed Tracing Output + +``` +iroha-config: Loaded `~/config.toml` + Configured: + `iroha.public_key` + `iroha.private_key` + `sumeragi.block_time_ms` + ... +iroha-config: Sourced ENV variables + Configured: + `torii.addr_p2p` from `API_ENDPOINT_URL` ENV var + ... + Overriddes: + `sumeragi.block_time_ms` from `BLOCK_TIME_MS` ENV var + ... +iroha-config: Final user configuration + ... (print full configuration, including applied default values) +``` + +#### Enabling Configuration Tracing + +To activate configuration tracing, users can use the `--trace-config` command-line argument or set the +`IROHA_TRACE_CONFIG=1` environment variable. Given the sequential nature of the configuration resolution process, the +tracing mechanism is intentionally separated from Iroha's main logger. This ensures that tracing can be performed even +before the configuration (which defines parameters like `logger.log_level`) is fully resolved. Keeping configuration +tracing as a straightforward on-off switch, without the complexity of log levels, further simplifies the user's +interaction and guarantees that all relevant trace data is captured when enabled. + +#### Sensitive Information + +While the trace provides detailed information, caution should be taken to avoid exposing sensitive data, such as private +keys, ensuring security isn't compromised. + +#### Further Integration + +Incorporating these traces into configuration-related error messages can further streamline troubleshooting, enhancing +user experience. (Refer to: [Proposal 6 - Exhaustive Error Messages](#proposal-6---exhaustive-error-messages)) + +#### Summary + +This proposal aims to allow users to see how the config gets pieced together, making it easier to set up and fix any +hiccups along the way. + +#### See Also + +- [[suggestion] Trace configuration parameters resolution · Issue #3502 · hyperledger/iroha](https://github.com/hyperledger/iroha/issues/3502) + +### Proposal 8 - Dual-View Configuration Resolution + +The dichotomy between configurations which are user-friendly and those that are efficient for software interpretation +and validation can be bridged through a dual-view configuration system: "User Config" and "Resolved Config". + +#### Objective + +Provide a distinction between the configuration as perceived by the user and the software's operational, validated +configuration view, striking a balance between user-friendliness and software efficiency & correctness. + +#### Rationale + +1. **User Clarity and Software Efficiency:** Keeping a representation that directly mirrors the reference for users + (User Config) and a separate, robustly-typed structure for the software's operational use (Resolved Config) benefits + both users and developers. +2. **Parse, Not Validate:** Transitioning from the User Config to the Resolved Config involves rigorous parsing and + validation. Upon the formation of the Resolved Config, it's inherently valid, minimising the likelihood of runtime + configuration errors. +3. **Domain-Specific Considerations:** The transformation phase applies domain (in this case, Iroha) specific rules and + logic, ensuring the final Resolved Config is both syntactically and semantically apt. + +#### Research about Resolved Configuration + +##### Redundant Fields + +In the background about [redundant fields](#redundant-fields), it's mentioned that these fields were made available to +end users primarily because the user config wasn't separated from the resolved config. However, in a two-stage +configuration resolution, these redundant fields can be managed more efficiently. + +##### Root-Level `public_key` and `private_key`[^1] + +When users provide their configuration input, it should be possible for them to specify only the private key. From this, +the public key can be derived. If both keys are provided, the system should derive the public key from the given private +key for validation. A discrepancy between the derived public key and the provided one would lead to an error. + +##### Telemetry Configuration + +Considering the user config's structure: + +```rust +struct Telemetry { + /// The node's name to be seen on the telemetry + name: Option, + /// The url of the telemetry, e.g., ws://127.0.0.1:8001/submit + url: Option, + /// The minimum period of time in seconds to wait before reconnecting + min_retry_period: u64, + /// The maximum exponent of 2 that is used for increasing delay between reconnections + max_retry_delay_exponent: u8, + /// The filepath that to write dev-telemetry to + file: Option, +} +``` + +While this flat structure is convenient to represent the user configuration of telemetry (i.e. the content of +`telemetry` section in the configuration file), there are some possible concerns: + +- If `name` is omitted, but some of the other fields are set, the system could warn that telemetry will not start unless + `name` is provided. +- If the `name` is given, but neither `url` nor `file` are specified, the system should either issue a warning or + terminate, signalling that the telemetry configuration is incomplete. +- If both the `url` and `file` fields are filled (indicating a potential conflict), a warning about the conflict should + be raised, followed by a system termination. + +> **Note**: Although telemetry might potentially have multiple emit targets, and the `file` and `url` fields might not +> need to be mutually exclusive in a real-world application, for the purposes of this example, let's assume that they +> are mutually exclusive. + +These points might be taken into account during the _resolution_ stage. A _resolved type_ for telemetry configuration +might avoid these conflicts by design: + +```rust +struct TelemetryResolved { + name: String, + target: TelemetryResolvedTarget, + min_retry_period: u64, + max_retry_delay_exponent: u8, +} + +enum TelemetryResolvedTarget { + Url(Url), + File(std::path::PathBuf), +} +``` + +#### Summary + +This proposal introduces a distinction between user-centric configuration (User Config) and the system's operational +perspective (Resolved Config). By separating these views, it facilitates clearer user inputs, streamlines +domain-specific validation, and aims to achieve a more robust and accurate software implementation of the configuration. +The goal is to ensure that the system operates optimally while still accommodating user input effectively. + +#### See Also + +- [[suggestion] Separate "user" config from "resolved" config · Issue #3500 · hyperledger/iroha](https://github.com/hyperledger/iroha/issues/3500) + +## Implementation Plan + +### Step 1 - Configuration Design & Reference Creation + +**Description**: This phase focuses on designing the configuration, considering user needs, best practices, and ensuring +alignment with project requirements. + +**Resources & Prerequisites**: + +- Comprehensive understanding of the current Iroha 2 configuration. +- Knowledge of best practices in configuration design. +- Collaboration with technical writers who will serve as the primary authors of the reference. +- Collaboration with developers and potential users to gather requirements and feedback. + +**Specific Objectives**: + +- Re-design naming of configuration parameters, introducing aliases for the ENV. +- Employ TOML as a domain language to describe the configuration. +- Establish a deprecation and migration policy as part of the reference. + +**Deliverables**: + +- A detailed configuration reference document outlining the structure, default values, data types, and descriptions for + each configuration parameter. + +**Potential Risks**: + +- Misalignment with user needs if not adequately consulted. +- Overcomplication or oversimplification of configuration design. + +**Feedback Loop**: + +- Regularly consult with both developers and potential users to ensure design alignment with their expectations and use + cases. + +### Step 2 - Development of a Generic Configuration Library + +**Description**: Develop a robust configuration library to act as the foundational piece for Iroha 2 and potentially +other projects due to its open-source nature. + +**Resources & Prerequisites**: + +- Might be done concurrently with [Step 1](#step-1---configuration-design--reference-creation) +- Proficient Rust developers with experience in procmacros. +- Understanding of requirements from the Configuration Design phase. + +**Specific Criteria**: + +- Support for exhaustive error messages, including batch-error collection and dependable deserialization failure + messages. +- Capabilities to compose configurations from incomplete parts for merging. +- Customizable merge strategies on a field-level. +- Support for multiple configuration source formats, notably TOML. +- Consider best and worst practices used in existing configuration-related libraries (namely: + [`config`](https://docs.rs/config/latest/config/), [`figment`](https://docs.rs/figment/latest/figment/), + [`schematic`](https://docs.rs/schematic/latest/schematic/)). + +**Deliverables**: + +- A well-documented, open-source configuration library. +- Unit and integration tests to ensure functionality. + +**Potential Risks**: + +- The library might not cater to all specific needs of Iroha 2. +- Potential technical debt if rushed or not adequately tested. + +**Feedback Loop**: + +- Continuous integration and testing setup. +- Regular checks with the Configuration Design team. + +### Step 3 - Iterative Implementation & Testing + +**Description**: Implement the designed configuration reference using the developed library. Ensure its functionality +through iterative testing. + +**Resources & Prerequisites**: + +- The configuration reference from [Step 1](#step-1---configuration-design--reference-creation). +- The configuration library from [Step 2](#step-2---development-of-a-generic-configuration-library). +- Testing environment and tools. + +**Deliverables**: + +- Fully implemented configuration for Iroha 2 according to the reference. +- Comprehensive tests showcasing the configuration's functionality and adherence to the reference. + +**Potential Risks**: + +- Implementation might diverge from the design if not regularly checked. +- Unforeseen technical challenges or limitations. + +**Feedback Loop**: + +- Regular testing and validation against the configuration reference. +- Potential beta testing or feedback sessions with a subset of users to understand usability and issues. + +### Step 4 - Integration, Field Testing & Feedback Collection + +**Description**: Integrate the newly developed configuration with Iroha 2. Conduct field tests to assess its real-world +performance and gather feedback. + +**Resources & Prerequisites**: + +- Completed and tested configuration from [Step 3](#step-3---iterative-implementation--testing). +- Access to Iroha 2 codebase and integration tools. +- A pool of testers or users for field-testing. + +**Deliverables**: + +- Configuration successfully integrated into Iroha 2. +- Feedback report from field tests detailing user experience, potential issues, and improvements. + +**Potential Risks**: + +- Integration challenges due to unforeseen incompatibilities. +- Negative user feedback or usability challenges. + +**Feedback Loop**: + +- Regular integration testing and validation. +- Direct communication channels with field testers to gather timely feedback. + +### Step 5 - Iterative Refinement & Optimization + +**Description**: After integrating the new configuration into Iroha 2, there might be some elements that need refining +based on user feedback and field-testing. + +**Resources & Prerequisites**: + +- Feedback from the [integration phase](#step-4---integration-field-testing--feedback-collection). +- Development and testing environments. + +**Deliverables**: + +- A refined configuration addressing user feedback. +- Documentation updates reflecting changes made. + +**Potential Risks**: + +- Over-optimization leading to reduced clarity. +- Potential rework if feedback indicates major design flaws. + +**Feedback Loop**: + +- Continued consultation with users. +- Performance isn't a primary concern, but the configuration should be robust and user-friendly. + +## Impact Assessment + +The proposed overhaul of Iroha 2's configuration system stands to significantly impact various facets of the project, +both technical and user-facing. These implications touch on the project's future scalability, user experience, and +maintainability. + +### Technical Impact + +- **Code Complexity**: The pursuit of an enhanced user experience may lead to a more intricate configuration codebase. + This increased complexity might make the system harder to maintain and debug. +- **Performance Impact**: While the configuration happens at startup and isn't a recurring process, there might be a + slight performance dip due to the enhancements. This is considered acceptable given the trade-off for improved user + experience. + +### Operational Impact + +- **User Transition**: Given the main goal of this RFC — simplifying the configuration for end users — we anticipate a + minimal migration pain for current users. The proposed changes, informed by earlier sections of this RFC, are intended + to streamline and simplify the process for them. +- **Deployment & Migration**: The revamped configuration could bring new deployment methods or procedures. Clear + documentation will be essential to guide system administrators and users through these changes. + +### User Impact + +- **Enhanced User Experience**: The main drive behind these changes is to offer users a more intuitive configuration + mechanism, reducing barriers to adoption and use. +- **Communication & Feedback**: Regular interaction with the community, especially the early adopters, will be crucial. + This helps to ensure that user feedback is accounted for, leading to a more refined final product. + +### Documentation & Training + +- **Updated Documentation**: Every change made to the configuration system should be meticulously documented. Updated + release notes and configuration guides will serve as primary resources for early adopters. +- **Training Sessions**: While intensive tutorials might not be needed, organizing small workshops or webinars for + significant changes can be beneficial, especially to ease any potential transition concerns. + +### Financial & Time Impact + +- **Resource Allocation**: The development of a new configuration library, alongside other proposed changes, will + necessitate both time and expertise. Efficiently managing these resources is vital to maintaining the momentum of the + project. +- **Project Timeline**: While we haven't set a specific timeline or deadline for the release, it's essential that the + configuration overhaul doesn't become a hindrance to the broader project's progress. + +### Risk Assessment + +- **Anticipating Challenges**: As the project progresses, unforeseen challenges related to the proposed changes may + arise. Having strategies to address these in advance ensures minimal disruption. +- **Backup Strategies**: Given the scope and depth of the changes, it's prudent to have contingency plans. These plans + ensure the project continues to progress smoothly, even when faced with unexpected hurdles. + +In essence, this RFC's primary objective is to drastically simplify Iroha 2's configuration for its users. Careful +planning, consistent communication, and regular feedback loops will be pivotal in achieving this objective successfully. + +## Conclusion + +Throughout this RFC, we've discussed fundamental enhancements for the Iroha 2 configuration system. From transitioning +to TOML and refining naming conventions, to creating an early configuration reference and clearer error messages, each +proposed change aims to elevate the user experience. Adopting these suggestions can transform Iroha 2's configuration +from a potential stumbling block to a robust foundation. We eagerly anticipate community feedback to refine these ideas +further. + +[^1]: +Please note that the configuration of keys (`public_key` and `private_key`) is a subject with its own set of +challenges and ongoing discussions. We are evaluating options to simplify and improve this area of configuration. +For further context, please refer to the following issues: + + - [Representation of key pair in configuration files · Issue #2135 · hyperledger/iroha](https://github.com/hyperledger/iroha/issues/2135) + - [Move keys out of configuration · Issue #2824 · hyperledger/iroha](https://github.com/hyperledger/iroha/issues/2824) + - [Guard against secrets leakage · Issue #3240 · hyperledger/iroha](https://github.com/hyperledger/iroha/issues/3240) + + Input on this topic is valuable. From f89aff51901362918e395c081d3c88132bc3a39d Mon Sep 17 00:00:00 2001 From: Dmitry Balashov Date: Wed, 20 Sep 2023 14:47:33 +0700 Subject: [PATCH 2/3] feat: apply minor changes - Include dev-only fields - Include migration tool mention - Include human-readable types proposal - Remove "dual view" proposal as extra Signed-off-by: Dmitry Balashov --- 000-config-overhaul.md | 275 ++++++++++++++++++----------------------- 1 file changed, 119 insertions(+), 156 deletions(-) diff --git a/000-config-overhaul.md b/000-config-overhaul.md index b7376f1..74d40e7 100644 --- a/000-config-overhaul.md +++ b/000-config-overhaul.md @@ -320,9 +320,9 @@ will simplify user interaction and align Iroha better with the broader Rust ecos #### See Also -- [Proposal 3 - Consistent Naming of Configuration Parameters](#proposal-3---consistent-naming-of-configuration-parameters) +- [Proposal 3 - Reconsider Parameter Naming Conventions](#proposal-3---reconsider-parameter-naming-conventions) - [Proposal 4 - Better Aliases for ENV](#proposal-4---better-aliases-for-env) -- [Proposal 5 - Define Deprecation and Migration Policy](#proposal-5---define-deprecation-and-migration-policy) +- [Proposal 6 - Define Deprecation and Migration Policy](#proposal-6---define-deprecation-and-migration-policy) ### Proposal 2 - Reference Before Implementation @@ -346,12 +346,12 @@ that any new features or changes are well-documented, understandable, and in ali - **[Proposal 1 - Use TOML](#proposal-1---use-toml)**: Settling on TOML as the standard configuration format before diving into the coding phase will prevent format-switching complications. -- **[Proposal 3 - Consistent Naming of Configuration Parameters](#proposal-3---consistent-naming-of-configuration-parameters)**: +- **[Proposal 3 - Reconsider Parameter Naming Conventions](#proposal-3---reconsider-parameter-naming-conventions)**: Prior to implementation, deciding on naming conventions and potential changes (like nesting keys under `iroha` or changing parameter names) will streamline development and ensure consistency. - **[Proposal 4 - Better Aliases for ENV](#proposal-4---better-aliases-for-env)**: Defining environment variable aliases in advance allows for more predictable and user-friendly configuration handling. -- **[Proposal 5 - Define Deprecation and Migration Policy](#proposal-5---define-deprecation-and-migration-policy)**: +- **[Proposal 6 - Define Deprecation and Migration Policy](#proposal-6---define-deprecation-and-migration-policy)**: Establishing guidelines on how to handle deprecated features or migrations ensures smoother transitions and fewer surprises for users. @@ -367,50 +367,68 @@ definitions, Iroha 2 will be better positioned to deliver a configuration system - [[suggestion] Get rid of "rustisms" in the configuration reference · Issue #3505 · hyperledger/iroha](https://github.com/hyperledger/iroha/issues/3505) - [[suggestion] Enhance configuration reference format · Issue #3507 · hyperledger/iroha](https://github.com/hyperledger/iroha/issues/3507) -### Proposal 3 - Consistent Naming of Configuration Parameters +### Proposal 3 - Reconsider Parameter Naming Conventions #### Objective -Standardise the naming conventions of Iroha's configuration parameters, enhancing their intuitiveness and consistency -for improved user interaction. +The objective of this proposal is to establish a consistent and intuitive naming convention for configuration +parameters, as well as to separate 'developer-only' fields from regular ones. #### Rationale -- **Clarity:** Consistent naming reduces ambiguity, allowing users to more easily understand and predict parameter - names. -- **Ease of Documentation:** A unified naming convention simplifies the documentation process, ensuring coherence and - easier updates. -- **Error Prevention:** Predictable and clear naming reduces the risk of configuration errors by users. +1. **Unified Naming Scheme**: Adopting `snake_case` uniformly across all configuration parameters will bring about a + more straightforward and less error-prone user experience. +2. **Developer-Only Fields**: The introduction of nested `[.dev]` sections will isolate parameters primarily + used for development purposes, reducing the likelihood of unintentional settings adjustments by end-users. +3. **Specific Renamings**: Some existing field names are either misleading or not fully representative of their + functionality. By renaming these, we enhance clarity and maintainability. -#### Proposed Name Changes +#### Developer-Only Sections -Here are the proposed changes for more consistent naming: +Introduce `[.dev]` sections to isolate 'developer-only' fields. These won't be included in the user +configuration reference but will be documented separately. -- `torii`: - - Rename from `p2p_addr` to `addr_p2p`. - - Rename from `api_url` to `addr_api`. - - Rename from `telemetry_url` to `addr_telemetry`. -- `logger`: - - Rename from `max_log_level` to `log_level` or simply `level`. -- `wsv`: - - Rename from `wasm_runtime_config` to `wasm_runtime`. +``` +# Old +[sumeragi] +debug_force_soft_fork = true + +# New +[sumeragi.dev] +force_soft_fork = true +``` -For the sake of uniformity, it might be beneficial to nest the root `public_key` and `private_key` under an `iroha` -namespace, resulting in `iroha.public_key` and `iroha.private_key`[^1]. +#### Specific Renamings -Please note that this list is not exhaustive. During the configuration reference design process, we might encounter -further areas of improvement and additional renaming suggestions, which shall be reflected as updates to the RFC. +This section details specific field renamings aimed at enhancing clarity, accuracy, and consistency. The changes are as +follows: + +- **Socket Addresses**: Replace the `url` keyword with `address` for fields that represent socket addresses (e.g., + `localhost:8080`) to avoid confusion. Therefore: + - `torii.api_url` becomes `torii.api_address` + - `torii.telemetry_url` becomes `torii.telemetry_address` +- **Key Configuration**: Simplify `iroha.public_key` and `iroha.private_key` to `public_key` and `private_key` + respectively, eliminating redundant namespace information. (footnote: [[80ad19 Footnote - Keys Configuration]]) +- **Telemetry Split**: Separate telemetry configuration into distinct namespaces for better isolation and clarity: + - `telemetry.substrate`: This will include `name`, `url`, `min_retry_period`, and `min_retry_delay_exponent`. + - `telemetry.file-output`: This will only include the `file` field. +- **Network Configuration**: Move `sumeragi.p2p_addr` to `network.address` for more contextual naming. +- **Logger Configuration**: + - Change `max_log_level` to either `log_level` or simply `level` to clarify its purpose. + +**Implementation Note**: This RFC may not be updated to reflect all new parameters or renamings that may appear during +the implementation process. #### Summary -This proposal aims to standardise the naming conventions within Iroha's configuration parameters, ensuring clarity and -predictability, which will enhance user experience and streamline documentation efforts. +This proposal unifies naming to `snake_case`, separates dev-only fields, and improves field clarity. Further changes may +not be reflected in this RFC. #### See Also - [Proposal 1 - Use TOML](#proposal-1---use-toml) - [Proposal 4 - Better Aliases for ENV](#proposal-4---better-aliases-for-env) -- [Proposal 5 - Define Deprecation and Migration Policy](#proposal-5---define-deprecation-and-migration-policy) +- [Proposal 6 - Define Deprecation and Migration Policy](#proposal-6---define-deprecation-and-migration-policy) - [Representation of key pair in configuration files · Issue #2135 · hyperledger/iroha](https://github.com/hyperledger/iroha/issues/2135) ### Proposal 4 - Better Aliases for ENV @@ -432,9 +450,9 @@ experience. #### Proposed Changes 1. **Rename Based on Functionality:** Shift from code-based naming to function-based naming. For example: - - `TORII_API_URL` becomes `API_ENDPOINT_URL`. - - Convert `KURA_INIT_MODE` to `BLOCK_REVALIDATION_MODE`. - - `LOGGER_MAX_LOG_LEVEL` becomes `MAX_LOG_LEVEL` or simply `LOG_LEVEL` + - `TORII_API_URL` becomes `API_ENDPOINT_URL`. + - Convert `KURA_INIT_MODE` to `BLOCK_REVALIDATION_MODE`. + - `LOGGER_MAX_LOG_LEVEL` becomes `MAX_LOG_LEVEL` or simply `LOG_LEVEL` 2. **Document All Aliases:** Clearly list main variables and their respective aliases in the configuration reference. 3. **Trace Resolution in Logs:** For clarity, ensure that when an alias is used, its resolution to the actual parameter is evident in logs. @@ -447,10 +465,47 @@ to make Iroha 2's configuration process more intuitive and error-free. #### See Also - [Proposal 2 - Reference Before Implementation](#proposal-2---reference-before-implementation) -- [Proposal 3 - Consistent Naming of Configuration Parameters](#proposal-3---consistent-naming-of-configuration-parameters) -- [Proposal 7 - Trace Configuration Resolution](#proposal-7---trace-configuration-resolution) +- [Proposal 3 - Reconsider Parameter Naming Conventions](#proposal-3---reconsider-parameter-naming-conventions) +- [Proposal 8 - Trace Configuration Resolution](#proposal-8---trace-configuration-resolution) + +### Proposal 5 - Human-Readable Types + +#### Objective + +The objective of this proposal is to enhance the user experience by allowing the specification of certain configuration +parameters using human-readable types, specifically for durations and byte sizes. + +#### Rationale + +1. **Enhanced Readability**: Using human-readable strings such as "1m 5s" for durations and "5MB 4KiB" for byte sizes + enhances the readability of configuration files, making them more accessible for non-technical users. +2. **Reduced Error-Prone Nature**: Human-readable types are less prone to mistakes compared to using raw numerical + values, which often require users to manually calculate conversions (e.g., minutes to milliseconds, megabytes to + bytes). + +#### Proposed Changes + +1. **Durations**: Allow duration fields to be specified in a more human-friendly format. Rather than requiring + milliseconds as an integer (e.g., `305000` for 305 seconds), support string inputs like `"5minutes 5seconds"`. +2. **Byte Sizes**: Similarly, for fields representing data sizes, support string inputs like `"4KiB"` or `"5MB"`, as an + alternative to just raw byte integers. + +```toml +# Old +cache_size = 5242880 +timeout = 1500 + +# New +cache_size = "5MB" +timeout = "1s 500ms" +``` + +#### Summary + +This proposal introduces human-readable types for duration and byte-size fields, making configuration more intuitive and +less error-prone. The changes will require updates to the parsing logic of the configuration system. -### Proposal 5 - Define Deprecation and Migration Policy +### Proposal 6 - Define Deprecation and Migration Policy #### Objective @@ -486,7 +541,7 @@ alternatives or configurations. Introducing a structured deprecation and migration policy not only establishes trust with users but also ensures the software evolves efficiently, balancing innovation with stability. -### Proposal 6 - Exhaustive Error Messages +### Proposal 7 - Exhaustive Error Messages #### Objective @@ -499,24 +554,24 @@ Diagnosing and resolving configuration errors can be challenging without detaile messaging system offers the following benefits: 1. **Parsing with Precision:** - - **Highlight Invalid Data's Location:** Pinpoint the exact location of errors, making it easier for users to - address. - - **Identify Missing Fields:** Rapidly flag any necessary but absent data, speeding up the configuration process. - - **Spot Unknown Fields:** Alert users to any unrecognized fields, helping them to quickly catch and fix - typographical mistakes or redundant information. + - **Highlight Invalid Data's Location:** Pinpoint the exact location of errors, making it easier for users to + address. + - **Identify Missing Fields:** Rapidly flag any necessary but absent data, speeding up the configuration process. + - **Spot Unknown Fields:** Alert users to any unrecognized fields, helping them to quickly catch and fix + typographical mistakes or redundant information. 2. **Clear Directions for Domain-Specific Errors:** - - **Highlighting Semantic Issues:** Inform users about errors or warnings tied to the higher-level domain details - specific to Iroha. This ensures that users are aware of any nuances or complexities related to Iroha itself, - allowing for a more refined and accurate configuration. + - **Highlighting Semantic Issues:** Inform users about errors or warnings tied to the higher-level domain details + specific to Iroha. This ensures that users are aware of any nuances or complexities related to Iroha itself, + allowing for a more refined and accurate configuration. 3. **Eliminate Guesswork:** - - **Full Transparency in Reporting:** Ensure users are immediately and fully informed of any issues, preventing - oversights and minimizing confusion. + - **Full Transparency in Reporting:** Ensure users are immediately and fully informed of any issues, preventing + oversights and minimizing confusion. 4. **Consistency in Communication:** - - **Uniform, Informative Messaging:** Maintain a consistent style across error messages, bolstering clarity and user - understanding. + - **Uniform, Informative Messaging:** Maintain a consistent style across error messages, bolstering clarity and user + understanding. 5. **Streamline Multi-Error Solutions:** - - **Bulk Reporting of Missing Fields:** If several fields are missing, display them all simultaneously for more - efficient troubleshooting. + - **Bulk Reporting of Missing Fields:** If several fields are missing, display them all simultaneously for more + efficient troubleshooting. #### Potential Solutions & Samples @@ -532,7 +587,7 @@ efficient configuration process for Iroha users. - [[suggestion] Enhance configuration parsing with precise field locations · Issue #3470 · hyperledger/iroha](https://github.com/hyperledger/iroha/issues/3470) -### Proposal 7 - Trace Configuration Resolution +### Proposal 8 - Trace Configuration Resolution #### Objective @@ -582,7 +637,7 @@ keys, ensuring security isn't compromised. #### Further Integration Incorporating these traces into configuration-related error messages can further streamline troubleshooting, enhancing -user experience. (Refer to: [Proposal 6 - Exhaustive Error Messages](#proposal-6---exhaustive-error-messages)) +user experience. (Refer to: [Proposal 7 - Exhaustive Error Messages](#proposal-7---exhaustive-error-messages)) #### Summary @@ -593,102 +648,6 @@ hiccups along the way. - [[suggestion] Trace configuration parameters resolution · Issue #3502 · hyperledger/iroha](https://github.com/hyperledger/iroha/issues/3502) -### Proposal 8 - Dual-View Configuration Resolution - -The dichotomy between configurations which are user-friendly and those that are efficient for software interpretation -and validation can be bridged through a dual-view configuration system: "User Config" and "Resolved Config". - -#### Objective - -Provide a distinction between the configuration as perceived by the user and the software's operational, validated -configuration view, striking a balance between user-friendliness and software efficiency & correctness. - -#### Rationale - -1. **User Clarity and Software Efficiency:** Keeping a representation that directly mirrors the reference for users - (User Config) and a separate, robustly-typed structure for the software's operational use (Resolved Config) benefits - both users and developers. -2. **Parse, Not Validate:** Transitioning from the User Config to the Resolved Config involves rigorous parsing and - validation. Upon the formation of the Resolved Config, it's inherently valid, minimising the likelihood of runtime - configuration errors. -3. **Domain-Specific Considerations:** The transformation phase applies domain (in this case, Iroha) specific rules and - logic, ensuring the final Resolved Config is both syntactically and semantically apt. - -#### Research about Resolved Configuration - -##### Redundant Fields - -In the background about [redundant fields](#redundant-fields), it's mentioned that these fields were made available to -end users primarily because the user config wasn't separated from the resolved config. However, in a two-stage -configuration resolution, these redundant fields can be managed more efficiently. - -##### Root-Level `public_key` and `private_key`[^1] - -When users provide their configuration input, it should be possible for them to specify only the private key. From this, -the public key can be derived. If both keys are provided, the system should derive the public key from the given private -key for validation. A discrepancy between the derived public key and the provided one would lead to an error. - -##### Telemetry Configuration - -Considering the user config's structure: - -```rust -struct Telemetry { - /// The node's name to be seen on the telemetry - name: Option, - /// The url of the telemetry, e.g., ws://127.0.0.1:8001/submit - url: Option, - /// The minimum period of time in seconds to wait before reconnecting - min_retry_period: u64, - /// The maximum exponent of 2 that is used for increasing delay between reconnections - max_retry_delay_exponent: u8, - /// The filepath that to write dev-telemetry to - file: Option, -} -``` - -While this flat structure is convenient to represent the user configuration of telemetry (i.e. the content of -`telemetry` section in the configuration file), there are some possible concerns: - -- If `name` is omitted, but some of the other fields are set, the system could warn that telemetry will not start unless - `name` is provided. -- If the `name` is given, but neither `url` nor `file` are specified, the system should either issue a warning or - terminate, signalling that the telemetry configuration is incomplete. -- If both the `url` and `file` fields are filled (indicating a potential conflict), a warning about the conflict should - be raised, followed by a system termination. - -> **Note**: Although telemetry might potentially have multiple emit targets, and the `file` and `url` fields might not -> need to be mutually exclusive in a real-world application, for the purposes of this example, let's assume that they -> are mutually exclusive. - -These points might be taken into account during the _resolution_ stage. A _resolved type_ for telemetry configuration -might avoid these conflicts by design: - -```rust -struct TelemetryResolved { - name: String, - target: TelemetryResolvedTarget, - min_retry_period: u64, - max_retry_delay_exponent: u8, -} - -enum TelemetryResolvedTarget { - Url(Url), - File(std::path::PathBuf), -} -``` - -#### Summary - -This proposal introduces a distinction between user-centric configuration (User Config) and the system's operational -perspective (Resolved Config). By separating these views, it facilitates clearer user inputs, streamlines -domain-specific validation, and aims to achieve a more robust and accurate software implementation of the configuration. -The goal is to ensure that the system operates optimally while still accommodating user input effectively. - -#### See Also - -- [[suggestion] Separate "user" config from "resolved" config · Issue #3500 · hyperledger/iroha](https://github.com/hyperledger/iroha/issues/3500) - ## Implementation Plan ### Step 1 - Configuration Design & Reference Creation @@ -761,10 +720,11 @@ other projects due to its open-source nature. - Continuous integration and testing setup. - Regular checks with the Configuration Design team. -### Step 3 - Iterative Implementation & Testing +### Step 3 - Iterative Implementation, Testing, and Migration Tool Development -**Description**: Implement the designed configuration reference using the developed library. Ensure its functionality -through iterative testing. +**Description**: Implement the designed configuration reference using the developed library. Concurrently, develop a +migration tool to help users seamlessly transition from the old to the new configuration system. Ensure the +functionality of both through iterative testing. **Resources & Prerequisites**: @@ -775,6 +735,7 @@ through iterative testing. **Deliverables**: - Fully implemented configuration for Iroha 2 according to the reference. +- A functional migration tool capable of converting older configurations to the new format. - Comprehensive tests showcasing the configuration's functionality and adherence to the reference. **Potential Risks**: @@ -785,7 +746,8 @@ through iterative testing. **Feedback Loop**: - Regular testing and validation against the configuration reference. -- Potential beta testing or feedback sessions with a subset of users to understand usability and issues. +- Potential beta testing or feedback sessions with a subset of users to understand usability and issues with the new + configuration and migration tool. ### Step 4 - Integration, Field Testing & Feedback Collection @@ -794,7 +756,8 @@ performance and gather feedback. **Resources & Prerequisites**: -- Completed and tested configuration from [Step 3](#step-3---iterative-implementation--testing). +- Completed and tested configuration from + [Step 3](#step-3---iterative-implementation-testing-and-migration-tool-development). - Access to Iroha 2 codebase and integration tools. - A pool of testers or users for field-testing. @@ -901,9 +864,9 @@ from a potential stumbling block to a robust foundation. We eagerly anticipate c further. [^1]: -Please note that the configuration of keys (`public_key` and `private_key`) is a subject with its own set of -challenges and ongoing discussions. We are evaluating options to simplify and improve this area of configuration. -For further context, please refer to the following issues: + Please note that the configuration of keys (`public_key` and `private_key`) is a subject with its own set of + challenges and ongoing discussions. We are evaluating options to simplify and improve this area of configuration. + For further context, please refer to the following issues: - [Representation of key pair in configuration files · Issue #2135 · hyperledger/iroha](https://github.com/hyperledger/iroha/issues/2135) - [Move keys out of configuration · Issue #2824 · hyperledger/iroha](https://github.com/hyperledger/iroha/issues/2824) From 0b38e1e051ed09a3a9b4acdb1de4c1db06fbeb94 Mon Sep 17 00:00:00 2001 From: Dmitry Balashov Date: Mon, 2 Oct 2023 15:46:17 +0700 Subject: [PATCH 3/3] chore: assign `001` number Signed-off-by: Dmitry Balashov --- 000-config-overhaul.md => 001-config-overhaul.md | 0 1 file changed, 0 insertions(+), 0 deletions(-) rename 000-config-overhaul.md => 001-config-overhaul.md (100%) diff --git a/000-config-overhaul.md b/001-config-overhaul.md similarity index 100% rename from 000-config-overhaul.md rename to 001-config-overhaul.md