"Seegrid will be due for a migration to confluence on the 1st of August. Any update on or after the 1st of August will NOT be migrated"

AdX3 Discussion

Contents

Related pages



Why

Most of the Adx3 planning seems to have occurred in teleconferences and other meetings which means the points debated are not visible to people doing implementation.

Now that we have a strawman schema up and some start on mapping tools, we need a more visible place for discussions.

This page is for discussing the what should be in AdX3 (which is fairly well established we hope but maybe subject to change).

For discussion of how to migrate see AdxMapping2to3Discussion.

Optimal History model Discussion

Questions based on samples

2006-03-16 ALS Chemex generated from spreadsheet

can we still track an Order of processing, or will this just be an optional "MeasureParameter" associated with each event ?

-- ColinLegg 16 Mar 2006

Order of processing can be tracked using the event model.

Each ProtoEvent or ProtoEventParam can have properties (from adxProcessing.xsd):

<group name="ProtoEventOptionalProperties">
    <sequence>
        <element name="time" type="om:TimeObjectPropertyType" minOccurs="0">
            <annotation>
                <documentation>Matches the om:time in om:ProcedureEventType</documentation>
            </annotation>
        </element>
        <element name="responsible" type="gmd:CI_ResponsibleParty_PropertyType" minOccurs="0">
            <annotation>
                <documentation>matches the om:responsible in om:EventType
                        
                        Person or organisation responsible for the event, if applicable. 
                The nature of the responsibility (i.e. the role of the party with respect to the event) may be indicated using the xlink:arcrole attribute. 
                Examples of roles are operator, sponsor, requestor, provider, processor, etc.</documentation>
            </annotation>
        </element>
        <element name="measureParameter" type="swe:TypedMeasureType" minOccurs="0" maxOccurs="unbounded">
            <annotation>
                <documentation>Stands in for om:measureParameter in Events. For standardised chains, put the params in the ProtoEvent instances. When params vary, override with measureParameter properties added to the ProtoParam instances inside the ProcessingHistory.</documentation>
            </annotation>
        </element>
    </sequence>
</group>
-- AndyDent - 17 Mar 2006

2006-03-15 BasicReport1.xml hand-prepared

Why not rename "Specimen Preparation Activity" to "Sample Processing Activity" and add a "Measurement" Activity Type, with "Instrument Type" as an attribute, plus a sub component detailing the "Analyte Suite" for that "Measurement Activity" with all it's extra attributes (Upper, Lower Det etc.. Then point to that "Measurement Activity" from the "Proto Measure Event". Then remove "dic2". -- ColinLegg 16 Mar 2006

  1. as defined in AdX3Glossary, I'm keeping Specimen as an informal term we can use to refer to things within the event chain and any activity is about converting Samples to such things.
  2. I lean towards retaining the idea of the definition of the procedure as something separate from the events:
    1. you can point to procedures defined in a separate file (dic2 is just a convenience)
    2. the definitions of the procedures can exist in a file with nothing being measured

-- AndyDent - 17 Mar 2006

Things to Discard

Stuff from the previous classes that we may want to discard as it can be replaced by the results of the virtual events.

adxSpecimen properties

  • preservative
  • matrix
  • media
  • status
  • comment

Support Types

  • specimenStatusCode - codes such as EMT = Empty container no specimen

Summary

History-oriented

  • Every measurement occurs at the end of a history of preparations and decisions (such as to repeat from a given point).
  • This history will be sufficiently rich to enable meeting any design goals.
  • The history may not necessarily be stored in a fully expanded form, taking advantage of uniform processing where possible.

Scalability

Be able to handle the minimalist case of a small lab which does not record any deviations or details other than the individual measurements produced.

...through to...

A fully-exploded individual history created for every single sample going into the lab, where the entire chain of custody is recorded with timestamps.

Where do Events live?

adx:ProcessingHistory will normally
  1. reference a standard history in protoEvents
  2. contain an optional set of parameters as a sequence of ProtoParams to add details to events in that history.

Thus you can generate fully-detailed unique event histories from the above as a transformation.

However, given a standard history which is only used by one adx:ProcessingHistory, there is no need to store it separately nor to have any extra parameters:

adx:ProcessingHistory may in this case contain a history unique to itself in protoEvents, composed of Events rather than protoEvents.

Linking ProtoParams to ProtoEvents

The current linking, as discussed at the meeting on 3/Mar/06, is by an ordinal number indicating a given ProtoParam matches the Nth ProtoEvent in the ProtoHistory used by the adx:ProcessingHistory.

eg:
The third <nop>protoEvent in:
  <gml:definitionMember>
    <adx:ProtoHistory gml:id="A_31">
      <gml:name>Path A 31</gml:name>
      <adx:protoEvent>
...
      <adx:protoEvent>
...
      <adx:protoEvent>
        <adx:PulpPrepProtoEvent gml:id="A_31H3">
          <gml:name/>
          <adx:procedureUsed xlink:href="#crush1"/>
        </adx:PulpPrepProtoEvent>
      </adx:protoEvent>

matched by lookup from the ord property of

  <adx:param>
    <adx:PulpPrepProtoParam gml:id="abc123_a3">
      <adx:ord>3</adx:ord>
      <adx:time>
        <gml:TimeInstant gml:id="abc123_3t">
          <gml:timePosition>2006-01-25T09:16:00</gml:timePosition>
        </gml:TimeInstant>
      </adx:time>
      <adx:massPrior uom="kg">2.43</adx:massPrior>
      <adx:massPost uom="kg">0.24</adx:massPost>
    </adx:PulpPrepProtoParam>
  </adx:param>

I am concerned this may not be robust and it makes more sense to store the id of the relevant .

-- AndyDent - 14 Mar 2006

Implementing

Flyweight pattern

One of the original Design Patterns from the book.

We are using this pattern where we want to store additional parameters to be able to flesh out a standard series of steps.

links:

Writing out in ADX3

Based on writing the Python script to process the ALS Chemex example (also available in an Excel version)

  1. iterating down each sample row:
    1. identify the starting chain of events as types of events
      1. the type of initial event - ReceiveSampleEvent for normal rows, based on the initial samples
      2. the unique preparation sequence, including the default values as measureParamsters, so the need for extra params is reduced
      3. any different values in the quality control columns
      4. anything else anyone can think of plaintive enquiry ?
    2. now iterating within each procedure across the row, that has data:
      1. take the starting chain of events from above and add steps for:
        • instrument preparation
        • tray assignment, if any tray numbers available
        • measurement
      2. generate a unique key for that chain
      3. retrieve a matching standard chain from dictionary, or add new standard chain
      4. generate a list of additional data to flavour the standard events, if there are any variations
      5. create an adx:ProcessingHistory referencing that standard chain
      6. create adx:Assay objects for each measured value belonging to that adx:ProcessingHistory
  2. repeat the row iteration process for the extra rows comprising repeats and standard runs, using the type of row to determine the initial event
    • RepeatEvent for general repeats at any stage
    • PulpReuseEvent for something started from pulp (re-running the entire sequence)

Reading in ADX3

Outdated Specimen-oriented Discussion

Does chain-of-custody thinking require use to track specific instruments?

If we are trying to trace back some contamination or other evidentiary review with the complete chain of custody, what happens if a (large) lab has multiple instruments which may be used for a given procedure?

Should this be recorded somehow?

Main.ColinLegg said in teleconf that this is not a scenario we need to worry about - labs do not track use of individual, identical machines.

-- AndyDent - 07 Mar 2006

SpecimenAnalysisRun Refactoring

This started as an email to ask a question, turned into a debate with myself and I think ended up answering its own original question and questions implied during the writing, so I figured I'd put it here as a design history.

This came out whilst trying to finish applying the new model to the ALS Chemex example spreadsheet.

From discussions with Colin last week it seems there's a clear concept we've named SpecimenAnalysisRun - the combination of the AssaySpecimen and a series of AssayProcedures run on that specimen.

This corresponds to a row in the typical spreadsheet. It provides a very clear place to record the fact that a given set of tests were rerun for some reason and a concrete object with which many users can identify.

Part of the reasoning behind this was that it allowed simplification of the Assay values - the measurements which were pointing to both AssaySpecimen and AssayProcedure. They could just point to a SpecimenAnalysisRun, as seen in AdX3SchemaUMLModel which (at time of writing) incorrectly shows the SpecimenAnalysisRun pointing to a single AssayProcedure.

However, if the SpecimenAnalysisRun contains more than one reference to an AssayProcedure then we won't know which procedure resulted in a given measurement. Oops!

This leads us back to the Assay still needing to point to the AssayProcedure.

In that case, although SpecimenAnalysisRun is a meaningful concept to the end user, I'm no longer 100% sure it actually represents something worth putting in the XML file. Its mere existence can be computed from the set of Assays.

The existing AssayProcedure plus AnalyteSensitivies covers the setup of the instruments to do the measuring. The other thing that needed adding was a set of steps representing the instrument-specific processing (eg: digestion, forming beads) which can be described by a unique AssayProcedure.

It all comes back to the defaults.

For each Instrument, there's a AssayProcedure with preparation steps such as digestion using various parameters. There is a small set of such procedures within a batch, probably just one per instrument.

An instrument's acceptance criteria may be varied on later runs - this can be described by another AssayProcedure with different AnalyteSensitivities and again there is a small number of these (usually still just one per instrument but maybe varied).

Assay measurement results are drawn from the pair of these - instrument-specific preparation plus a procedure with sensitivies.

This could be described by a separate object - an InstrumentRun .

Do we need InstrumentRun or will extending AssayProcedure be OK?

  1. Adding InstrumentRun provides a clear point at which we say a given AssayProcedure and its preparation are combined. It recognises that the preparation steps such as digestion are a defined sequence that may often be independent of the sensitivies used to measure.
  2. Extending AssayProcedure to include the preparation steps keeps the number of objects smaller but may muddy concepts.
  3. Using a separate concept of InstrumentRun also gives us a place for specific instrument identifiers (asked above).

-- AndyDent - 06 Mar 2006

SpecimenAnalysisRun as Event?

In the new model should SpecimenAnalysisRun be an Event?

That seems to fit - it has a time location and duration and would fit the batch processing idea.

-- AndyDent - 06 Mar 2006

Failure to receive a specimen

How is this handled? Should a adx:SpecimenReceiptEvent be generated with special time code, or does failure to have a adx:SpecimenReceiptEvent indicate the specimen was not received?

-- AndyDent - 26 Feb 2006

ColinLegg (?) suggested in teleconf that the specimen should just not appear at all in the XML file.

In working on the ALS Chemex translation I noticed that the adx:SpecimentTypeCode in the adx:status allows for this with SNR as Specimen Not Received so it appears such a specimen could be handled by having an adx:specimen with such a status code but no adx:SpecimenReceiptEvent.

-- AndyDent - 27 Feb 2006

Specimen Processing Details other than Prep

adx:Specimen ...sa:processingDetails is a required element, the annotation for which says "May contain collection, sampling & preparation procedures". Are we concerned with anything other than preparation procedures at this stage? Do the source documents include information from which collection and sampling procedures can be inferred?

-- RyanFraser 27 Feb 2006

Specimen Processing - Procedure History

Given that a om:ProcedureHistory has to include a series of om:step items, why does it have an om:time property as well and, more importantly, what does that time property signify?

Is this a question better asked over in ProceduresAndInstruments?

-- AndyDent - 27 Feb 2006

Event Chains and Shorthand

Based on our teleconf Monday morning, it was agreed we should have a way to refer to a standard sequence for a specimen other than splits (which was implemented as Simon suggested - adding a simple xlink:href on the sa:processingDetails as seen in BasicReport1.xml on the AssaySpecimen abc123_a.

However, we also discussed that for any split specimens, a full event chain would be required which means a complete history in sa:processingDetails (as seen in abc123_b) and a SpecimenPreparationEvent for each step in that chain.

This is about 40 - 60 lines of XML per split speciment, depending on the number of preparation events.

Assuming 10% splits and remembering that a split implies at least TWO chains of fully documented events, for every 100 initial specimens we will be adding about 1200 lines of XML for the event documentation.

Whilst these events can potentially contain unique information, I think it's worth dicussing how they could be simplified if there are no unique values. One possibility is referring to pre and post-split sequences rather than duplicating the events.

-- AndyDent - 27 Feb 2006

Am very wary of establishing too many ways of doing the same thing. This was what caused confusion in ADX2 between provider and consumer. When developing the "history"-based approach to describing splits, the primary reqirementwas to allow for the fact that the current methods (based on codes and special words) was confusing between different companies and providers, was potentially ambiguous, particularly for non-standard prep histories, and was definitely not scalable. IMHO anything that takes us in the direction of introducing yet another syntax smacks of taking us in a proprietary and ultimately unsustainable direction.

Is "lines of code" in fact the key metric here? If we loose lines of code, but at cost of losing the gains from the explicit approach, then we have not really moved forward.

-- SimonCox - 28 Feb 2006

Ideas for 3.1

(yes, already)

Explaining Why Instances don't match Declarations

 
Topic revision: r19 - 15 Oct 2010, UnknownUser
 

Current license: All material on this collaboration platform is licensed under a Creative Commons Attribution 3.0 Australia Licence (CC BY 3.0).