Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Project Tracking: Instrumentation Stability + Semantic Conventions Working Group #2753

Closed
jsuereth opened this issue Aug 26, 2022 · 24 comments
Closed
Assignees
Labels
area:semantic-conventions Related to semantic conventions project-tracking triaged-accepted The issue is triaged and accepted by the OTel community, one can proceed with creating a PR proposal

Comments

@jsuereth
Copy link
Contributor

jsuereth commented Aug 26, 2022

Description

Solve the technical needs required for Semantic Conventions and Instrumentation Stability, as outlined in this proposal.

Project Board

Pending TC Approval & Sufficient staffing/momentum

Deliverables

At the highest level, the group will be delivering (or improving existing status-quo of) the following:

  • Telemetry Definition: Ensuring a common machine-readable format for defining signals produced in instrumentation, and associated tooling for human usage and development.
  • Telemetry Stability: Improving compatibility definition beyond just addition of signals and clarifying x-signal compatibility of attributes for Resource, Instrumentation Scope and
  • Telemetry Evolution: Expanding existing telemetry schema features and unifying with Definition/Stability work.
  • Semantic Conventions: Outlining a repeatable process for marking semantic conventions as stable that allows domain experts to drive key decision.

See the proposal for details

Staffing / Help Wanted

This project is currently in proposal mode. We'd like to leverage this issue to advertise and attract interested parties.

Required staffing

Projects cannot be started until they the following participants have been identified:

  • @jsuereth Project Lead (+ TC member)
  • @reyang Second TC sponsor
  • @tigrannajaryan schema-file change reviewer / contributor
  • Need: Signal experts (metrics/logs/traces) willing to work on data-model + signal compatibility
  • Need: Engineers willing to write prototypes / work on tooling
    • Semconv buid-tools:
      • RST/MD docgen
      • Jinja / code-gen (x-language)
      • Go / OTel Collector code-gen
      • Integration w/ language build tooling
  • @lmolkova signal expert + tooling contributor
  • @Mpdreamz signal expert + tooling contributor
  • @trask java codegen contributor
  • Maintainers willing to approve prototypes
    • @cijothomas .NET Maintainer + Prototype approver
    • @lalitb C++ Maintainer + Prototype approver
    • @lzchen Python Maintainer + Prototype approver
    • @MSNev JavaScript Approver

Meeting Times

Once a project is started, the working group should meet regularly for discussion. These meeting times should be posted on the OpenTelemetry public calendar.

Timeline

This project is feature-driven vs. time driven. However, given the press of semantic convention CLs, an improvement on telemetry stability + definition are needed within 2022. Additionally, any timeline will need to reflect overall staffing + interest across the community. An aggressively optimistic road map:

  • 2022 Q4
    • Improve specification around telemetry stability, focusing on x-signal issues.
    • A unified telemetry definition format with documentation + codegen tools.
  • 2023 Q1
    • Unification of Telemetry Schema + Telemetry definition
    • First semantic convention marked stable
  • 2023 Q2
    • Expansion of Telemetry Evolution
    • Repeatable process for semantic convention definitions
    • Incremental improvements to build tools

Labels

We propose to repurpose existing labels in addition to adding two more:

  • area:semantic-conventions: This now denotes expert working group issues around defining semantic conventions (the process, not the tooling). Additionally, this is (and should continue) to be used in tandem with "semconv:expert-area" labels.
  • area:telemetry-tooling: Issues and request around the semconv codegen or docgen and related tooling (go's own build-tool, e.g) belong here or directly in the appropriate repository.
  • area:telemetry-stability: This denotes issues and concerns around defining telemetry-schema, available migrations, the metadata format for specifying telemetry, etc.

The later two labels would be the immediate focus of this project and working group.

Linked Issues and PRs

Blocked

Unresolved issues to tackle by this project

TODO(jsuereth): Attach relevant tooling/semconv/stability PRs

@tigrannajaryan
Copy link
Member

@open-telemetry/instr-wg FYI.

@tigrannajaryan
Copy link
Member

@jsuereth An OTEP that is probably relevant to be in the Linked Issues list: open-telemetry/oteps#199

@reyang reyang added the triaged-accepted The issue is triaged and accepted by the OTel community, one can proceed with creating a PR proposal label Aug 26, 2022
@reyang reyang self-assigned this Aug 26, 2022
@lmolkova
Copy link
Contributor

This is another relevant issue:

@lmolkova
Copy link
Contributor

I'd love to help with

  • data-model + signal compatibility
  • Semconv buid-tools

@jsuereth
Copy link
Contributor Author

Thanks @lmolkova!!! I've added you to the staffing section.

@cijothomas
Copy link
Member

I am a maintainer for .NET SIG, and can sign up for reviewing any prototypes in .NET.

@cijothomas
Copy link
Member

I am a maintainer for .NET SIG, and can sign up for reviewing any prototypes in .NET.

Slight correcting : As maintainer, I am anyway expected to review all PRs. What I meant to say is - I can prioritize reviewing any PRs directly contributing towards Semantic Convention/Instrumentation stability

@jsuereth
Copy link
Contributor Author

Thanks @cijothomas !!! added.

@tigrannajaryan
Copy link
Member

tigrannajaryan commented Aug 26, 2022

I'd like to help/continue working on Schema File related stuff. I am not sure how much time I can allocate to this, but at least I want to review all relevant PRs, even if I cannot submit them myself.

@reyang
Copy link
Member

reyang commented Aug 26, 2022

I've put myself as the Second TC Sponsor and assigned this issue to myself.

@jsuereth
Copy link
Contributor Author

@tigrannajaryan Understood. Added you as specifically interested in schema-files. Appreciate the caveat on time and definitely don't want you to deviate from your current responsibilities here.

@reyang Thanks much for signing up as sponsor!

@trask
Copy link
Member

trask commented Aug 26, 2022

I'd love to be involved with this and can sign up to write any codegen targeting Java (whether the codegen itself is written in Java or not)

@lzchen
Copy link
Contributor

lzchen commented Aug 26, 2022

I am a Python maintainer and would be interested in reviewing prototypes in Python.

@MSNev
Copy link

MSNev commented Aug 26, 2022

I can sign up as well, I'm a JS approver and a JS Web sandbox Maintainer and a member of the Client RUM Sig

@lalitb
Copy link
Member

lalitb commented Aug 26, 2022

I am a C++ maintainer, and if it helps, I can be a member to review/approve prototypes in C++.

@Mpdreamz
Copy link

Mpdreamz commented Sep 8, 2022

I can sign up as well for the following parts:

  • Signal experts (metrics/logs/traces) willing to work on data-model + signal compatibility

Especially in the context of open-telemetry/oteps#199 can help with classification of which fields work well for different types signals and backends

  • Engineers willing to write prototypes / work on tooling

I've lead similar specification work to drive codegen @ Elastic, can definitely help scope efforts.
Most likely won't have time to actually put my hands on the keyboard to spin out a prototype though.

@reyang
Copy link
Member

reyang commented Sep 8, 2022

@Mpdreamz thanks! I've updated the issue description by adding your name. Which programming language would you prefer for prototyping?

@jsuereth
Copy link
Contributor Author

Thanks to all folks who are interested. Please fill out this doodle of your preferred meeting time, ideally before friday as I'll be scheduling our first WG call next week.

Note: We are only selecting the timeslot not the specific date. The goal is for this meeting to happen every other week until the task list is complete.

@hughesjj
Copy link

I'm currently working to develop some go receivers and have prior experience with jinja+python, would love to join the committee.

@jsuereth
Copy link
Contributor Author

For folks working on semantic conventions. Here's a set of guidelines for avoiding what we expect are breaking changes:

  • Attributes defined for traces/logs that may be shared w/ metrics will likely have cardinality restrictions applied to them going forward.
  • Attributes/Resource definitions will likely be rejected until we have better refinement on Resource/Entity (see Refine which attributes of Resource contribute to Metric Identity. #2775)
  • Metrics definitions that mirror existing structure a likely to be ok (i.e. the report against a similar resource as today, the have similar labels). There may be changes to what RESOURCE a metric reports against going forward.
  • There may be some changes that occur to match (or have simple mapping to/from) elastic common schema. This is less likely to affect metrics, but may affect traces/logs/event semantics.
  • At some point, uniformity / machine-readable format will be required for all semantic conventions for all signals. Any table for metric/logs should match existing structure or update ALL tables with new fields + justification of the new field. Be warned, this means additional scrutiny and may be worth discussing separately from the semconv change itself.

@lsl-2020
Copy link

lsl-2020 commented Nov 7, 2022

Hi experts, could anyone share the latest updates on this project please? My team is waiting for the official version of instrumentation libraries to unblock roll out to production. Thanks!

@dmitryax
Copy link
Member

dmitryax commented Nov 7, 2022

I can join the effort as a code owner of the collector metrics metadata format and the metrics builder:

Go / OTel Collector code-gen

BTW I recently proposed guidelines and stability guarantees for that part of the collector, in case anyone is interested: open-telemetry/opentelemetry-collector#6358

@joaopgrassi
Copy link
Member

@lsl-2020 you can check the meeting notes to see what was discussed in the last meeting: https://docs.google.com/document/d/10xG7DNKWRhxNmFGt3yYd3980a9uwS8lMl2LvQL3VNK8/edit

@tedsuo tedsuo changed the title [Project Tracking] Instrumentation Stability + Semantic Conventions Working Group Project Tracking: Instrumentation Stability + Semantic Conventions Working Group Jan 9, 2023
@austinlparker
Copy link
Member

Closed as no longer relevant due to refactoring of semconv SIGs

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area:semantic-conventions Related to semantic conventions project-tracking triaged-accepted The issue is triaged and accepted by the OTel community, one can proceed with creating a PR proposal
Projects
Status: Completed Projects
Development

No branches or pull requests