From: Denise A Troll ZIG MEETING - DAY ONE October 2, 1996 Brussels, Belgium Meeting convened by Erik Lorenz Petersen, Chairman of the EWOS Expert Group on Library Applications and also outgoing Chairman of EFILA. Introductory comments were made by Jacques Rega, Director of EWOS, European Workshop for Open Systems. EFILA meeting Thursday morning 8:00-9:00. [Ed. note relayed from host: EFILA is an activity funded jointly by the European Commission and EWOS. The new Chairman of EFILA is Gordon Pedersen.] MH: "A ZIG meeting is a cross between a socker game and anarchy." He invited everyone to speak up, join in, holler louder, etc. DT will take minutes. STATUS REPORTS Will be posted in a separate message. REVISIONS TO AGENDA * CL: move item 3 on the agenda to tomorrow - There is a printed handout available from the working group that attendees should read before the discussion. * Guidelines for V4 is new business. * Postpone item 6 until tomorrow. * Add issue of ASN.1-1990; include support for ISO-646. * BW: status of ISO adopting Z39.50. * Extensions pending to Item Order. * Extensions to BIB-1 attribute set. * BM: brief update on CIMI profile and other profiles. * RL: will cover other profiles at SIGLA meeting. ZIG LIST OF ATTENDEES Will be distributed in email and posted with ZIG status reports. ZIG REGISTRY OF IMPLEMENTORS RD did his usual dance -- initial to verify, etc. The latest version of the registry is available on the Web. HTML OID (BW) Issue: BW proposed that there be an OID for HTML because HTML is ubiquitous enough to justify defining one. (The proposal derives from the record format required by the ZIG.) Discussion: BMc: what version of HTML and how do we specify it? BW: intentionally leave it fuzzy ...so the ZIG can follow what the rest of the world is doing. RD: has no official position on this, but unofficially he is very apprehensive -- what precedent does this set? and what are the implications for SGML? If we do this, RD wants to draw the line at HTML; he doesn't want this to extend to SGML. In the past we have agreed to carry SGML as an element in GRS-1. There is an OID for SDIF, but it is not clean enough to use. MH: SGML has its own OID elsewhere. RD: GRS was profiled in 10 lines to do ZSTARTS, but HTML is more complicated, e.g., is DTD implied or must it be represented? RD prefers solution using GRS to represent HTML. But some folks haven't done version 3. How do they ask for an HTML document in GRS-1? (?):HTML has diverged from SGML significantly, e.g., using forms. Having an OID for HTML would make it easy for users to request HTML. BW: we could make the ZIG definition point to ITF's already-existing definition. Outcome: The ZIG accepted the proposal. BW will provide prose that will (eventually :-) constitute the ZIG's definition of this OID for HTML. STRUCTURED VOCABULARY BROWSE (JK) Background: JK: the proposal for a structured vocabulary browse has been available for about a year now, but on the back burner. The challenge is to build a fully-functional Medline, i.e., to use Z39.50 to browse the hierarchical vocabulary of MESH subject headings. There are two goals: (1) to search for (assist in the discovery of) the correct terms to use in a subsequent database search (2) to browse (explore) thesaurii as an end in itself Z39.50 Scan is inadequate because it makes assumptions about ordering and (?). Issue: The proposed structured vocabulary browse involves a set of term nodes with static parent-child relationships, and dynamically-related nodes. The University of California has a prototype of this service providing access to MESH headings. Explain has not been implemented in the prototype, but the proposal has implications for Explain -- to allow the client to discover the name of the term database (structured vocabulary) that you want to search or browse. Discussion: BW: there is a MARC standard for thesaurii, i.e., a known record format and structure, that should be leveraged here. MT concurs. It seems like all we need is a way for Explain to convey the name of the thesaurus database. LW recapped an earlier discussion: at a previous meeting, we defined Scan in v3 to enable browsing and retrieval from thesaurii, and we defined attributes (e.g., narrower, broader); there are few flaws in the current standard for doing this. JK's proposal does have some enhanced capabilities, but there is no standardized structure for the data retrieved. JK: one of the reasons NOT to use Scan is because we want to deliver these rich (thesaurus) records in different structures or formats. Outcome: JK will provide prose on the ZIG list. RD wants the hierarchical browse modeled alng the lines of the collections profile ("are you still pursuing this along the lines of a digital collection?"). JK hasn't given it much thought. RD and JK will confer about this. The previous proposal did not specify the record syntax; the new proposal should. DEFAULT ELEMENT SET NAME (RD) Issue: RD: when you query the Explain database for element set names for a particular database, you get a list of element set names. One of these element set names should be designated as the default. RD proposed changing the Explain semantics, instead of changing the standard. The proposal would only apply to an AccessInfo structure that references a DatabaseInfo record, in which case the first element set name listed is the default. Discussion: LW: seems reasonable. RL: there's no mechanism to convey this semantic. (?): A comment could be added to the standard. (?): what if someone doesn't have a default? Wayne (?): Changing the semantics does change the protocol. Will any servers break? LW: the impact of this is minimal because there are currently so few v3 implementations. Outcome: RD will post the proposed change in semantics to the list and solicit comments. GUIDELINES FOR VERSION 4 (RD) RD proposed four guidelines for the development of Z39.50 version 4: GUIDELINE / ISSUE 1: RE-ALIGNMENT OF THE Z39.50 FACILITIES - RD: we don't need a facility for every service; facilitites need to be revised or removed. He proposed a reorganization of existing services within facilities (see handout or ZIG list posting). Discussion: MD and KT: we don't need facilities at all. CL: facilities were introduced for structuring the document and have no real significance for the protocol otherwise. Agreed. Outcome: No concluding or summary statement was made, but the clear impression was that the ZIG (a) did not think the concept of facilitites warranted spending precious time and (b) had no serious objections (at this time) to the idea of dispensing with the notion of facilities in version 4. GUIDELINE / ISSUE 2: DECOMPOSITION OF THE STANDARD - The official standard documentation continues to be unintelligible to many implementors or potential implementors, so RD proposed reorganizing or revising the document, e.g., moving object definitions, BIB-1, eSpec, GRS-1, etc. into external (companion) documents. Discussion: LW observed that it may be helpful to discern what is stable and what is not. He also expressed concern that it could be a problem locating companion documents to the standard. RL: on the other hand, the very size of the standard puts many developers off. CL suggested a different organizing principle for what remains part of the standard (as an appendix) and what moves out. For historical reasons, the standard includes lots of tools for specific applications of Z39.50, e.g., BIB-1, MARC and definitions of application-specific Extended Services. These tools could move out of the main standard and be maintained by domain experts; it's a disservice to include them as part of the standard. A noted exception is Explain record syntax, which is part of the core of the standard. Objection: BMc (?): the standard will be difficult to understand with only the core and no context. Shrinking the standard by moving much of the content into companion documents will create a different burden for tutorial material, since the standard will contain less information about building real applications. People need to be able to implement the standard with only the standard doc in hand; they shouldn't need additional docs to do it. Discussion: RD: but we currently have a problem selling Z39.50 to people who think it's unintelligible. MD: perhaps we should design different documents for different audiences, e.g., implementors, buyers, etc. -- to separate the issue of promoting Z39.50 from the task of implementing it. BW: we should also move out of the standard what has not already been implemented and tested (agreeing with LW). (?): since some of the definitions in Z39.50 are intended to be usable by other protocols, where appropriate, separating them from the standard may increase their utility. BM: the state of the implementation tends to be defined by actual experience, rather than the original document. JZ suggested dividing the standard into parts that can be progressed independently of the core standard. Lots of pieces (e.g., services identified by option bits) could be worked on independently. Not all items would be subject to balloting; some are merely subject to registration (OID-identified items). There was some concern about support for companion documents, but RD said they could be maintained by the Z39.50 Maintenance Agency and that NISO would be very happy to publish and sell all these documents. (?): things like attribute sets for specific information arenas are not best served by maintenance by bodies like NISO. RD: the whole issue of balloting needs to be addressed. Currently Z39.50 is balloted by NISO. This needs to be internationalized. Outcome: Discussion continued for a while, with no resolution to the problems -- though all agreed that the documentation of the standard remains a problem. GUIDELINE / ISSUE 3: RELAXING COMPATIBILITY WITH EARLIER VERSIONS - RD: Z39.50 1995 version may have gone overboard in maintaining compatibility with the v2 standard. This was necessary for Z39.50 acceptance, but resulted in significant growth of the standard. Version 4 might deprecate some parameters, that would disappear from the standard in v5. Discussion: There are two sides to this coin. KT: we can cut the redundancy, etc, but commercial products must support earlier versions to be successful. LW: we can remove un-used or redundant parameters, or deprecate things, but reducing compatibility can make implementation take longer; we must look at individual cases (e.g., the Group proposal for ZSTARTS instead of Piggyback). MD agrees with LW. Here the discussion went all over the map. [I couldn't keep track of all of the details.] There are trade-offs in the relative effort to implement the next version if you have to go back and tear out code that is currently compliant with the previous versions. We should focus on redundant, or unclear functionality, rather than removing functionality; if we proceed by degrees, we may not get there. DL: perhaps we should define v4 the way we want it, but also provide an ASN.1 set for a compatibility tool. MD: some of the things we are talking about are in fact used for practical purposes (e.g., Piggyback). Redefining these transactions using Groups would require redevelopment by implementors. The ZIG noted that there are still lots of v2 implementations, and plenty of folks just starting to deal with v3. BM: we cannot fix Init without creating an incompatibility; doing Groups without deprecating Piggyback would be wrong; carrying along baggage is not a good approach. JZ: it would be very dangerous to define a v4 implementation which couldn't talk to v2. We could use Init to negotiate v4. If v4 is not agreed to, then drop back to v2/v3. Market pressures will be brought to bear on incompatibility. CL: clients and servers are in different situations in terms of these economic factors. He thinks deprecating things in the standard is evil (it didn't work well with CCL) and encourages a case-by-case review of things, rather than a general, monolithic approach. What would we get out of relaxing compatibility with earlier versions? Performance gains; elegance; greater simplicity. We need to map out the objectives (positive aspects) for v4, rather than just talk about reducing the problems (negative aspects). RD mentioned segmentation as an example of advanced retrieval stuff that could be moved into a separate document for advanced implementations; backward compatibility ties might be cut in v5, not v4. Outcome: No concluding or summary statement was made. GUIDELINE / ISSUE 4: NEW FEATURES - RD would like to contain (resist) the addition of numerous or complex features. He recommended significant editorial procedures and some enabling features, such as the Group proposal, but trying to resist shoe-horning lots of features into v4. He proposed that v4 be a small incremental shift, not as big a jump as v3. Perhaps we~ll look at a blueprint draft of v4 at the next ZIG meeting. Outcome: There was little discussion, but no one seemed to object. So ended day one of the ZIG in Brussels, October 1996. ZIG MEETING - DAY TWO October 3, 1996 Brussels, Belgium ATTRIBUTE HANDLING (CL) Background: CL provided background on the current problems with attribute sets (discussed at the February 1996 ZIG). He distributed a handout (preliminary report) that briefly described the working assumptions and recommendations of the Working Group on Attribute Set Architecture that met at the Library of Congress in September 1996. The group is to develop guidance for domain experts to develop new attribute sets. The primary proposal is to define attribute classes and types; attributes within a class would have the same structure or be the same attribute type. There is a concept of a core attribute sets that would be inherited, and a basic set of use attributes. Attribute values will be both numeric and character strings, for ease of managing. There are many unresolved issues. For example: BIB-1 probably does not have enough attribute types and, consequently, overloads them. STAS provided good insights into additional attribute types needed. What about the ASN.1 typing for search terms? What about the format of the data -- should it be normalized? How do we balance the use of ASN.1 typing with the use of the format attribute? Some of the ZSTARTS work has indicated the need for attributes for query management (e.g., stop wording and term weights). Issue: What types of attributes should be defined? The working group proposed the following attribute types: * relation * language * character set * content authority * use * scope-of-use - a requirement for applications with sophisticated hierarchical markup, e.g., SGML -- to find a place name within an abstract * position * expansion / interpretation - a logical interpretation of values; could be used with stemming, morphological variants, part of speech, case sensitivity * format / structure * query management & execution. Discussion: Some alternatives were discussed, e.g., using nested use attributes (similar to tag path concept) instead of scope-of-use attributes. RD: some of these attributes may only be meaningful from client to server. CL: We need to agree on a way to permit mapping between element names and use attributes. The mapping should be extensible and include a name space for locally defined attributes (like local fields in MARC record) to avoid clutter and additional work. We also need to deal with the semantics of repeated or omitted attributes. And we need to define a core utility of attributes in each class; this work can probably accomplished in a couple more meetings of the working group. Thereafter, work on BIB-2 definitions should be based on the new architecture of attribute classes and types. RD raised the issue of a class of attributes for the query management attribute type. He also commented that some attributes only apply in response to a query, for example, a count attribute returned by a target. CL: it's an open question whether such attributes should be forbidden client to server, or just ignored. MH asked a question about scope-of-use attributes, which could be hierarchical: has any thought been given to building structure around attributes? Do we need meta-attributes? Can we ignore attributes? Is handling them mandatory or optional? CL responded that the working group had trouble coming up with examples of use attributes with a deep hierarchy, but agreed that deep hierarchies present coding problems in a flat name space. We would have to agree on how to encode hierarchical attribute structures. One of the next tasks for the working group is to think through the interplay between the type-1 query structure and the attribute definitions. KT asked a question about the development of BIB-2: what are NISO's plans? CL: the standards development committee has discussed this and is considering recommending a ballot to NISO to charter a committee on this. The thinking of NISO is that the scope of BIB-1 is murky; it began with bibliographic information, then was extended to A&I databases and beyond. NISO (BIB-2) would probably stick primarily to bibliographic information. Perhaps we need several attribute sets, e.g., one tied to MARC records. RD: how will the group working on BIB-2 be populated, i.e., will the ZIG be involved? RD thinks the ZIG should participate, but not lead. CL presumes that NISO will form a committee; NISO committee meetings are open, so there's plenty of opportunity for input. Note that the committee does not vote; the NISO membership votes. RF perceives many current use attributes as term attributes and is concered about too many types being introduced; we need to develop a taxonomy of attribute sets or principles for the development of attribute sets. CL: the working group attempted to develop a taxonomy inductively. In the past we got into trouble by having too few attribute types and overloading them. ASN.1 typing is a mixed blessing, since the ASN.1 parser gets involved in interpreting attributes. KT asked a questions about the type 102-query count attribute: if the server sends something to the client, should the client be send back an altered query with no mods? This is sometimes desirable. CL: and gives strong impetus to the notion of symmetry of attribute usage. Query handling should be symmetrical -- this is consistent with the ZSTARTS conclusions. (?): a class that has only one instance is a problem. CL: the working group will only define a single class right now (because we don't know what would be a good second class), but we will remain open for new realms of usage. Attribute sets that are consistent with the class that we're defining have certain consistent semantic rules. In the future, if we need a new attribute type, we shouldn't hack the existing class to get interoperability, but create a new class. An attribute class will have a list of types that are part of that class, but applications need not use all types within that class. JZ: some of the utility attribute sets like Explain and Extended Services might be different classes of attribute sets. JK asked a question about two-way communication. Attributes convey different information to the server, which then typically re-maps the query. We need more granular responses to specific attribute requests. This has not been addressed in detail. CL: some places two-way communication makes sense (in diagnostics like "I don't do right truncation"); other places it doesn't make sense. LW: we once had a concept with the type-102 query that what the server returned to the client was an interpreted query, including granular diagnostics in the query structure in the AdditionalSearchInfo from the target. CL: the query could carry detailed diagnostic information. People should be able to develop attribute sets as appropriate. The working group has and will continue to focus on attribute architecture. The working group did not carefully consider the type-102 query, but future work must do this. RL had a quibble with the complexity of character set attributes. There was much discussion about his quibble (KT, RD, CL) and a question about inheritance. Perhaps we should depend on the character escape sequences to announce the character set. Or perhaps we should support higher level use of the attributes in the tree of the query to indicate the character set of terms below that node. The issue is tied to the ASN.1 of the datatype of the term, and the version of ASN.1 that we we support. RL proposed abandoning scope-of-use attributes. If I have multiple use attributes, to which does the scope-of-use attribute apply? Discussion ensued (MH, LW, DL). There's at least a potential need to extend the database schema to the search function, as well as the retrieval function. This relates to the use of a TagPath concept vs. the scope-of-use concept. There is confusion about what we're looking for (e.g., a personal name) and the place we're looking (e.g., the author field). RD explained that "use" refers to the usage of the term, and "scope" refers to limiting where to look in the record -- the advantage is clarity. CL: there is a murky duality between the way in which a term is used and the places where it is used. He agreed that we may not need the notion of scope-of-use, but he was not convinced yet. MD observed: there are three things going on: (1) this is X, (2) do something with X, and (3) where was X used (schema). No resolution was reached regarding scope-of-use attributes. RL quibbled: ranging is not an operator, but a complex operand with attributes that apply across the whole thing. CL: that's an interesting idea, but he's not sure we're ready to jump to that generality; while elegant, that approach may be difficult to map to (?). BW agreed with RL: we need complex operands; a range should be treated as a complex operand. No concensus was reached. BW was confused by query management and asked for clarification: how will this be used for two-way communication? Does this belong in AdditionalSearchInfo? CL: the working group was trying to define attribute types, not necessarily populate all instances of that type. They focused on the type-1 query and did not consider how the attribute types fit with the type-102 query. BW: ZSTARTS is too complicated; no one's doing it. [There was much discussion here but I couldn't type fast enough to catch it all.] MD: by trying to normalize different aspects of attributes or attribute types, the working group is multiplying our problems, i.e., making it more difficult to describe what clients and servers are doing. He prefers the simpicity of fewer attribute types to the complexity of more attribute types, and focusing on the precision vs. the fuzziness of searching. CL: there's a "conservation of difficulty" problem here. How do you characterize what a given server will do? What are all of the legal combinations? The problem is that if we reduce the number of attribute types, then we must put upper limits on usage (7 of this type allowed, etc.). LW agrees with CL; the lack of precision in search results derives from vague (broad, general) attributes. We need more attributes to do precise searches. CL: as soon as you have repeatable attribute types, how do you repeat them? Do you "or" or "and" them? How many qualifiers do you have? What's the format for the value, etc.? MH: I'm hearing two different communities. One wants specificity. The other wants simplicity. Both are reasonable expectations. MD prefers international involvement in attribute sets; not just NISO (U.S.A.). CL suspects that NISO would welcome international participation and expertise in the drafting process. Drafting is different from maintenance. Maintenance might be anywhere. CL is not sure what the correct forum might be. NISO is willing to take this on. SR: Yes, people can participate in NISO meetings, but only NISO members can vote. Question of international membership in NISO? NISO considers all comments from both members and non-members. Whatever the case, BIB-2 has to be handled in an international setting, or at least be an international enterprise, not a national enterprise. CL: Where do we go from here? Two issues: ISSUE 1: HOW DO WE CONCLUDE THE ATTRIBUTE SET ARCHITECTURE? - CL proposed that the attribute-architecture working group should meet a couple more times to address the open issues, e.g., repeated instances of attributes in a query. The ZIG concensus was that the work of this group is fruitful and on the right track. CL said that anything that affects the query per se should be addressed by the ZIG, not in isolation by the Working Group On Attribute Set Architecture. He proposed November 18 for a working group meeting in Washington D.C., and probably another meeting after that before the end of the year. His goal is for the group to conclude its work in early 1997. BW: what form will the conclusions take? CL responded: * At least one document that basically describes the characteristics of the attribute class. * Probably a second document that compiles (a) basic recommendations of interest to a broader group and (b) the implications for the protocol. * Possibly some guidelines for attribute set designers. The working group should apply the architecture of classes and types to a set of attributes to explore their userfulness. They could examine STAS, BIB-1, geo-spatial attribute sets, etc. as a reality check on their architecture work. * The group should also prepare a straw man application of the model to illustrate the various attribute types, including examples of applying the new attribute types to problematic attributes and the resulting definitions that resolve the problems in the new architecture. KT: when will the group have something that can be tested? The answer to this question is important for resource allocation. CL: late first quarter 1997. CL will post an announcement of the next working group meeting on the ZIG list. ISSUE 2: WHERE ARE WE GOING WITH NISO AND BIB-2? How do we incorporate the international input and make it an international enterprise? RD: There will be a TC46 meeting in conjunction with the April ZIG in Washington D.C. that may be a context for coordinating BIB-2 activity in an international setting. EXPLAIN TESTBED (MD) Issue: MD: EWOS working with Ameritech, SilverPlatter, AT&T to implement and test Explain. CrossNet and (?) joined the group later, as part of the ONE project. Preliminary testing of Explain servers was conducted using the GLP test tool in March/April 1996, but they need a real client to test the servers. JG of Ameritech offered to implement a real Explain client to replace the test tool, but a real client is not yet available. To date there are no existing client implementations to test the Explain functionality; consequently the Explain testbed group is waiting for clients to conduct a real test. There are projects that are developing real v3 Explain clients (e.g., CrossNet, Index Data, the ONE project) to fill this gap. Simultaneous with Explain testing, some other work has gone forward. MD posted a report to the ZIG list, including the defects discovered in the standard. For example, the design of certain components of Explain is illogical. What is a "brief" Explain record? What is a "full" Explain record? What does this have to do with the size of the record? Next year we should have different Explain client implementations to conduct further (real) tests. MD suggested deferring a detailed discussion until the next ZIG, and keeping the testbed activity in its current form until real clients are available. In the meantime, MD will maintain a list of Explain implementors; he invited Explain developers to contact him to get on his list. He foresees a transition from the Explain testbed subgroup to an Explain implementors subgroup. MD invited folks to subscribe to the Explain listserv. He will report again at the next ZIG and post to the ZIG list every 2-3 months. Discussion: LW asked: will the forthcoming proposed changes to Explain be part of v4? RD: defects will be corrected in v3; defect reports will be discussed later at this ZIG. MD clarified: if there are problems with the Explain model, however, they become part of v4. LW asked: what is a "defect" and what is a "significant improvement." RD: problems with syntax are clearly defects; the fuzzy items require discussion. OTHER TESTBED ACTIVITIES MH: during status reports, many people said they were doing v3 servers and clients. Are there problems with interoperability? Do we need a v3 Explain or a v3 general testbed? People should post their v3 experiences to the list. CL reported that nothing has happened in the arena of general v3 testbed activity due to lack of time, and agreed to abandon the plans for a general v3 testbed if there is no need for it. RD didn't like the idea of abandoning the v3 testbed, fearing that ad hoc interoperability testing was inadequate -- but thought that maybe it shouldn't be coordinated through the U.S., since much of the v3 activity is happening in Europe. He called for a new general v3 testbed coordinator. Someone nominated BW to coordinate U.S. activity, but BW declined. CL: people are just beginning to work on this in the U.S. and it's not yet time to test interoperability. New systems in Europe are beginning with v3. The next thing to do is for people to post brief status reports of what they've done with v3 and the interoperability problems they're having. KT will post a relevant URL to the list. RD has added a link to the Z39.50 maintenance agency Web page about v3 testing and interoperability issues; send information about v3 interoperability and activity to RD so that he can link and track what's going on. SR reported on the Scan testbed. Scan has been reasonably well tested, though not formally. She will try to post a list of sites (clients and servers) that are willing to test interoperability of various versions or features. RD will post to the ZIG list inviting people to send him information about their Scan interoperability testing; he'll put this information and the list of sites on the Web. MESSAGE SIZE NEGOTATION (RD) Issue: RD explained that the proposal (see handout or message posted on ZIG list), which came about from work with ZSTARTS, is offered as an implementors agreement, not a change to the standard. Discussion: BW, MD suggested making preferred-message-size and exceptional-record-size optional in v4, rather than deprecating them. CJ asked a question about using negative numbers and the interpretation of zero (e.g., as uninitialized data). RD just said "no" to using negative numbers. Outcome: The ZIG accepted the proposal as an implementors agreement. Z39.50 CATALOGING PROFILE (Janifer) Issues: Catalogers need to search a wide range of databases: full-text, bibliographic, SGML, etc. The proposal on the table (which came about from work on a project at the national library of Australia and New Zealand) is to use the Z39.50 Extended Services Update service to update local and union catalogues. Phase one of the service, i.e., updating a database using Extended Service Update, will be available in a few months. (The project is using the National Library of Canada's client and tools.) The objectives at the ZIG meeting are: * to examine the general applicability of the proposed cataloging profile * to examine its conformity to Z39.50 * to examine implementation issues * to achieve international acceptance Bottom line: a profile that can only talk to one server is not a profile, but a proprietary interface. Implication: acceptance of the profile will change the role of union catalogues and provide catalogers with a single user interface (client) to update multiple servers. The folks who worked on the proposal want full integration of update and query, and effective error resolution. Why use Z39.50? Because Z39.50: * is designed for information search and retrieval -- the foundation of cataloging * is widely accepted * is extensible * has already defined an Update service GENERAL APPLICABILITY OF THE CATALOGING PROFILE The cataloging profile calls for several extensions to Z39.50 to meet cataloging-specfic requirements. These requirements are: * recognition of three parts to a database = three kinds of data: -- bibliographic, authority, and holdings information -- catalogers need to be able to say what kind of data to update in the Update request * detection of duplicates -- the target needs to detect a duplicate submittedby a client -- the target must be able to decide between a record insert, a record update, and a record replacement * review distribution - the origin could use the permission subfield to indicate if a record needs to be reviewed * concurrent updates - this is key to the cataloging profile: -- assume no record locking because the target has no idea how long an origin will keep a record and because records may go to multiple origins; use 005 (time-date stamp) that origin can't change and returns with modified record -- the target may sometimes need to send a surrogate record and a full record to tell the origin that it can't do what was requested * global changes - to multiple records at once -- to keep a database clean -- to control various versions of a distributed database -- use Element Update for this * merge - information from one record into another record (e.g., holdings information) -- a form of delete, but put elements of it in another record * inquiry (narrow search) by holdings -- only find books held in a certain library (done using an attribute) * validation and checking lists of codes (see profile Appendix 1) -- with validation, assume that the client could do what it liked, but that the target would never trust it -- how match to Z39.50 attributes? -- how does the target interpret and process this? * acquire authority MARC records * several additional BIB-1 attributes -- e.g., to handle thesauris and authority records (as opposed to bib records), or to limit retrieval to items published in a particular country Discussion: JZ suggested that they refer to the approved and proposed BIB-1 attribute extensions by the National Library of Canada (NLC), which cover essentially all values for MARC, as well as most of what is being suggested here. About 250 attributes were proposed by NLC, and no one has commented on them or objected to them -- so JZ assumes approval. Searching holding and authority records will include thesaurus databases, so we need attributes for these databases, as well. The draft proposal includes the mapping of fields to BIB-1 attributes, including searching the libraries' own holdings information. The profile will have problems if BIB-1 is replaced by a BIB-2 that looks only at bibliographic records. CONFORMITY TO Z39.50 Issues: The proposal is to use schema to distinguish cataloging, authority, and holdings data. It would require extensive use of correlation information and OtherInfo. To facilitate interoperability, the initial mandatory requirements are limited. The origin and target must do access control, record Insert and Update (don't change date and time), and have package support. Since there is a low probability of anonymous client-server relationships in cataloging, it is unlikely that these implementations would use Explain. Catalogers are biased towards MARC records, but the profile should not be MARC specific (e.g., it should work with SGML). [An additional point was made about error messages, but I didn't catch it.] Discussion: Someone questioned the need for record Update to require the full record. What is a full record? The client may not know the full record. The target should just be told what to do and apply these changes to the full record as it knows it. EASE OF IMPLEMENTATION Issues: The Z39.50 tools do not need to be updated to implement the profile. However, the ZIG does have little experience with Extended Services Update. Someone raised the question about who owns the profile. Discussion: SR: what's going on with holdings? Are we talking abut a separate MARC holdings record, or a field in a MARC bib record? Response: either _insert_ a MARC holdings record, or _update_ a MARC bib record. SR asked questions about diagnostic information, and requested clear language for the origin to process valid fields or subfields; this language should be made explicit in the profile. The profile currently does not include errors on holdings information. The next iteration should include this information. KT: what are you using access control for? Is this publicly available? Response: we use access control to validate the client. The project's client and server should be available in November 1996. RL: the target will need an access control diagnostic that says when the user is not authorized to do a certain function. JZ is terrified of global updates by a local client. Response: access control must control what local clients (users) can do on local databases and on union catalogues. RL requested record locking because many servers support or require it, and asked whether this is a specific profile for cataloging or a general update facility? He recommended a specific profile for cataloging. KT was concerned about the simultaneous updating of a local database and a union catalog -- what if one succeeds and the other fails? CL said that it was scary to see a federated update transaction packaged into Extended Services Update; he was "not eager to play this on a large scale and bet my operation on it." Are you assuming that all participating systems are running on a highly synchronized common clock? SD: the only time that matters is the source time; that time acts as a magic cookie and returns unchanged. CJ, MD commented that we need something outside of Z39.50 to handle inconsistencies, but in general the approach is good. RD asked a question about schema: how do you supply within the request what the record is? Something about the OID implying a schema that governs...but has never been articulated.... Is that sufficient? JZ: no, because several MARC record types have the same syntax; there should be a record schema OID for each MARC type, etc. CL agreed. MH: but MARC bib records also contain holdings. RD: where should this profile reside? If the dilemma is between the Asian OIW and the Z39.50 Maintenance Agency, RD wants the Maintenance Agency to have it -- even as a draft. He'll help take it through the standards process. RD expressed concern about coordinating efforts among those who are implementing database Update. If you are working on a project that's finding flaws or sees a need to extend this service, RD would like to facilitate collaboration. He also cautioned the project not to overload the OtherInfo parameter if the extended serviced really needs some additional features. He was not sure if global update was needed, but if it is, it shouldn't be in OtherInfo ("oh, by the way, do it everywhere"). LW was concerned about shortcomings discovered in database Update -- some are results of features that were originally requested, but then eliminated in the interest of simplicity. RD encouraged LW to advance these as defect reports. Outcome: MH said that the profile was a great piece of work. There was some interest from the ZIG, but he was surprised that there wasn't more, given the library and vendor community present at the meeting. Should someone establish a mailing list or leave the discussion on the ZIG list? The group agreed to keep it on the ZIG list. [lunch break] MH wrap-up on the cataloging profile: discussion will continue on the ZIG list. We will discuss the next iteration at the April 1997 ZIG meeting. The cataloging profile group will work with JZ on attributes. ZSTARTS open meeting tomorrow after the ZIG (12:00-1:00). Background: In the Spring 1996, RD became aware that the Stanford Digital Library project was developing an approach and protocol to do distributed searching and ranked retrieval. The project claimed that their approach was related to Z39.50, but RD investigated and found it gave only lip service to Z39.50. Further developments included a distributed indexing and searching conference at MIT. The ZIG was represented at the conference and contributed papers. RD's observation was that the real agenda was to validate the concept of indexing and the browsing and collection model as opposed to searching, though there was an attempt to narrow the issue to indexing. The STARTS (formerly called STAIRS) proposal and Z39.50 got lots of attention at the meeting, but the discussions were separate. There was some controversy about whether or not the Stanford project needed a standard protocol. The result was that we needed a rendering of Z39.50 that accomplished certain goals -- this rendering came to be called "Z-lite." The ZIG group was trying to understand why STARTS was not using Z39.50. Response: the standard is unintelligible. Nonetheless the conference raised consciousness about Z39.50 and the need to profile information retrieval over the Web or Internet. After the MIT conference, over the next 2-3 months the effort gained visibility. The Stanford team had assembled important companies and had achieved concensus on the requirements for this application of distributed searching and ranked retrieval ; these requirements became the target for the ZSTARTS profile. In July 1996, the discussion of Z-lite was posted to the ZIG list and quickly went beyond the scope of the profile. Mike (Hebron?) from Fulcrum strongly favored use of Z39.50; he was the motivator and champion of the initiative. Discussion continued at a two-day meeting in August with a small group of hand-picked vendors and Z39.50 implementors. An early document listed the requirements and procedures (i.e., how to use Z39.50 to solve the problems). After the August meeting, the group "dummied down" the initial document and began work on the ZSTARTS profile. Fulcrum wanted a profile by late August and implementations by the end of the year. RD said that the ZIG doesn't work that way. Folks have relaxed their expectations, but their is still strong urgency that this must progress quickly or the effort should be dropped. RD thinks the effort has a 50% chance of success, but nonetheless success would justify the effort. The next two-day meeting included more people, both people involved in the project (day one) and four commercial vendors (day two - MicroSoft, Fulcrum, OpenText and ?). (Netscape claimed a meeting conflict and was not represented at the meeting.) The result was concensus that the effort should continue; there needs to be a standard that addresses the requirements of the STARTS project (e.g., distributed searching, ranking, sorting). The suggested name for the next draft is Simple Ranked Retrieval Protocol (SRRP). Since that meeting, Verity has expressed interest. The next ZSTARTS meeting is tomorrow after the ZIG. What are the discussion points? What are the management issues? Discussion: MH made a comment about the overall goal of the project: people want to retrieve and assemble results from traditional Web services (with HTML markup) and Z39.50 services. This is a real opportunity to expand the accessibility of Z39.50 databases and to merge these two communities. LW agreed; it would be advantageous if more vendors supported this. Will it help if we (the customers) tell the vendors to support Z39.50? RD: yes, that's exactly what we should be doing. LW: will the process be opened up now? RD: the group was closed until mid-August, but it's open now. CL commented on LW's question about constituencies and markets: it important to understand that there are two constituencies involved here: (1) people marketing Web indexing databases (e.g., Altavista, Lycos) and (2) search engine manufacturers, who make engines that might underlie such systems. Web catalogers are queasy about this. They would like to merge results and rank them, but this is fundamentally at odds with their business model of selling advertising, which is where their revenue comes from. Engine manufacturers have a different interest. The motivations of the two groups conflict. RL: we need to acknowledge and apologize to the query type-102 folks; we're stepping on their toes, but have little choice because of the timing. RD talked to PR about it, and he seemed to understand. MD: the current profile does not conform to the Z39.50 standard. For example, it deletes parameters from the search request that are required by the standard. Is that intentional or an oversight? RD: The profile does not address everything (e.g., piggybacking), which is not to say that those parameters are taken out. The profile is not non-conformant, but it requires elaboration. He presumed that the ZIG would approve in principle some kind of "grouping" to address, not piggybacking, but pre-sorting. (Recall concatenation proposal from a few years back.) The pre-sort criteria is a requirement of STARTS that must be accommodated, though it is not currently addressed in the draft profile. To do that, we would be non-conformant -- hence the need for grouping. [The proposal for grouping was an agenda item later at the ZIG.] MD: the ASN.1 in the profile is incorrect. RD: that's an error; errors will be corrected. The intention is Z39.50 conformance. There may be holes that we missed, but the intent is for this to be a self-contained profile that minimizes the need for other documents. To what extent does Stanford support the profile? Stanford people attended the August meeting and edited the draft. But at the present time we do not know their exact stance, though we do know that they are not entirely abandoning their approach in favor of Z39.50. To what extent is WC3 or IETF involved? RD: We haven't made overtures to them yet. RD: implementations would probably be based on an HTTP interface to a meta-searcher Z39.50. MH: the model is unclear; it would have helped to have Netscape involved. LW: at the ASN.1 encoding level, will ZSTARTS be interoperable with Z39.50 applications? RD: yes (if we succeed). BW asked for a clarification of why sort is needed if results are ranked. RD: in addition to ranking, the Stanford project needs author sort, date sort, etc. MH tabled the discussion until tomorrow's ZTARTS meeting. CIP PROFILE (SM) CIP = Catalogue Interoperability Protocol CEOS = Committee for Earth Observation Satellites, which is developing standards and cataloging guidelines for interoperablity. The work is being done by multiple task teams, e.g., CINTEX (a testbed for interoperability work), Browse (not to be confused with Z39.50 Scan), Network, Protocol (works with CINTEX team). The objectives of the Protocol Task Team (PTT) are (a) to get administrative and technical agreement on how the protocol will work, and (b) to develop requirements and protocol specifications. SM showed a model of existing retrieval services in many CEOS systems, which include a directory of high level inventories, browse of thumbnails, etc. Overview of CIP - CIP is being driven by service providers, standards (including maintenance and flexibility to gain international acceptance), and the users' need for enhanced functionality (e.g., using one interface to order product data from multiple sources). They need to dynamically configure the user interface using Explain, and to provide administration and design guidelines so that CIP nodes can interoperate. The standard client-server approach did not fully support what was needed, so they're developing a retrieval manager (middleware) for such things as distributed searches across existing, different local systems. A CIP collection is new concept, contributing to the notion of a digital collection. A CIP collection is hierarchical data grouping of categorized types designed to improve the flexibility of data provision and to avoid confusion of terms. There are three types of CIP collections: provider archive collections (analogous to physical inventories), provide theme collections and user theme collections. Collections can overlap, have multiple parents, and have the potential for recursion. There are key differences between CIP collections and the digital collections profile. CIP collections are product descriptors. Collection searches use the retrieval manager to work with local systems. [Here I missed a point about CIP space.] CIP attributes derive from various attribute sets. Semantic attributes can be described using a core set of attributes. Development path: In March 1995, they developed a syntax and functional specification for CIP. In the Fall 1995, they investigated Z39.50 and based a specification on it. By Spring 1996, they had implemented a Z39.50 CIP prototype and produced a report. Their ideas were verified as reasonable and the project went forward. Now they are working on a demonstrator Web client, retrieval manager, and catalog translator. The demonstrator is to verify that the CIP approach works. It will entail three kinds of retrieval managers: one to route queries to other retrieval managers; one to act as a catalog server or small data provider; another to serve as a gateway to the server of a big data provider like NASA. Open issues: There is lots of work to be done on use attributes, combinations, schemas (for collections), diagnostics, etc. They must decide how to use Explain, and explore whether the CIP profile conforms to the Z39.50 standard. They also need collection maintenance and building tools, and rules for collections. The future: CIP runs to the end of 1997. The next iteration wil handle accounting, ordering, etc. They want to establish a federation of CEOS/CIP users, first in the United States, then internationally, and to mount a major program based on the CIP protocol. They want to build close, mutually beneficial contacts with the ZIG and are now a ZIG member. There is lots of enthusiasm among various agencies in Europe about CIP. Discussion: CIP entails two types of searches: (1) searching for information about a collection, and (2) searching for a product description within collections. There is no way in Z39.50 to distinguish the two types of searches, so CIP uses two externals. RD will incorporate CIP comments and concerns into the digital collection profile, but someone needs to summarize for him. Or should the ZIG use the CIP profile as a companion profile like CIMI? Response: the CIP group has little flexibility to diverge from what they're doing, though draft 7 of the digital collection profile does include significant points, as does the museum collection profile. They will discuss this further with RD. Is the CIP group cooperating with European environmental agencies? Good question. They have established a small study contract with (?) and are attempting to build some links, but it is a background priority. SQL EXTENSIONS TO Z39.50 (SF) As prelude to discussion of the proposed extensions to Z39.50, SF offered the following advantages of providing SQL queries to the Z39.50 community: * SQL would expand Z39.50 to a larger (business) community * SQL would provide more powerful querying capabilities * SQL would use existing RDMS catalogues And the followijng advantages of providing Z39.50 to the SQL community: * Z39.50 would expand the level of access to RDMS * Z39.50 would provide attribute sets and global schema * Z39.50 would provide a set of services to manage (?) The proposal (see handout) includes: * a new query type-SQL2 * a minor change to Scan (e.g., send me every 100th record) - on hold until they try to implement Scan * two additions to Explain syntax = versionInfo, dictionaryInfo Schemas in RDMS are not as stable as bibliographic catalogs, hence the need for versionInfo to record the history of the database schema. They plan to implement Explain in late 1996. * supporting two record syntaxes = SQL2-RS and SQL2-ERR (error messages not yet defined) Current status: They have implemented their proposed Z39.50 protocol extension. The ZINC prototype will be available by the end of November 1996. It currently includes Init, Present, Search and Close. By 1997 they plan to add Scan, Explain, and some Extended Services. Pitfalls include (a) trying to have multiple attribute sets for fields that are in the actual SQL statement, and (b) trying to avoid the ASN.1 for SQL statements, which is possible to do, but very complex and perhaps unnecessary. Discussion: RD commented that the Type-1 query is required for Z39.50 conformance to the standard, but SQL users would not use or want type-1, which raises conformance issues. JZ is concerned about the Z39.50 result set model and its correspondence to the results table from an SQL query, which has no notion of a record and dynamically creates the results table based on the specifications in the query. How would you do a subsequent query on that result set or display a record? You lose the notion of logical records that are maintained at the server. Z39.50 has a shared assumption of the element set. SF: Some applications would want global schemas and merged results, but to do that within an SQL query would require even more thought. RD: let's ignore the issue of scanning every 100th record, but what do you mean by "scan" a result set? SF: Does not want to scan an index, but a (?). CL: as a specification for how SQL interacts with Z39.50, the next iteration of the proposal needs to specify precisely the relationship between the results of the query and the specific result set name in the query PDU. When is the statement successful, when does it fail, and how does this relate to the result set? How do you preface an SQL query with an attribute? SQL only knows fields, columns, tables. SF: if your database supports SQL and Z39.50 attribute sets, the server would know and merge the results. CL: this assumes a canonical textual representation of attribute sets, which are currently numeric in Z39.50. JZ: Z39.50 requires attribute and value, but SQL allows only one. SF: yes, hence the need for this discussion. KT is baffled; what kinds of databases are you intending to use this with? If it's a salary database, there is no analog in ASN.1. SF: the demo is based on a census database. KT: two things are missing: (1) A way for the origin to specify to the target: do this query and organize the results in this table. There's nowhere in ASN.1 for this information to go. SF: a result set name would be assigned and sent along with the query; the result set name would be applied to a table. KT: this restricts result set names to table rules, which currently aren't included in Z39.50. (2) A way to do back referencing. SF thinks we can work around this. KT likes the dynamics of the fields returned. RL observed that all that Z39.50 is contributing to this is the concept of abstract schemas. If that's it, then a much closer approach is a translator between SQL and Z39.50 applications. SF asked how can you do the extended services then? How can you query and refine and refine and refine until you get something of reasonable size for viewing? How can you scan? What about periodic scheduling? They can't do any of this in SQL and don't want to reinvent the wheel. So ended day two of the Brussels ZIG meeting. ZIG MEETING - DAY THREE October 4, 1996 Brussels, Belgium FOLLOW-UP TO SQL PROPOSAL (SF) Discussion continued: RF: what's the point of combining mututally inoperable protocols with different goals or functions? SQL is data-processing and Z39.50 is information retrieval. SF sees Z39.50 as a higher level access protocol. She wants to ship queries, not SQL statements. [Discussion here included other protocols, e.g., IDA, that I don't know so I didn't understand.] (?): the ZIG cannot evaluate the proposal until the relationship between the result set and the records is verified. MH: the ZIG model is "extremely silent" about what's in a result set -- intentionally, to give BW leeway; there is no guarantee that the server will give you anything in a record, i.e., this relationship, whatever it is, does not break the protocol. (?): not certain that text and data are really distinct or separate communities; you often don't know what fields are in a database until you get there. [KT asked a question here about some query type, but I missed it.] CL: when we recast this, we need to clarify how this proposal interacts with concurrency control as defined in SQL. It looks like you're essentially going to serialize the database during the execution of the search. You need to be careful. SF: this is not clarified in the proposal or the protocol; she thought it was negotiated at the server. CL (looking at the ASN.1 in the proposal for returning results): it looks like SQL1; SQL2 introduces date-and-time data type, and SQL3 (if it gets standardized) is going to introduce a full type system. What problems do you foresee in the result record syntax? SF didn't address this. The proposal used standard SQL. The intention is to use the 92 standard and SQL3 if becomes standard. JK, CL: Z39.50 will let you be ambiguous about add-on searches; SQL will not. SQL has strong notions about record locking, duplicates, etc. There are no add-on searches, but a new query. Things like triggers may further complicate it. (?): reluctant to let the server decide what to do unless it has some way to convey this information. SF: that information would be stored in Explain. Outcome: MH: where do we go from here on this proposal? SF came tothe ZIG for comment and for help. RL: we owe SF a show of interest. What vendors are interested, should it be progressed, would it be used? A show of hands indicated substantial interest. Who will help? KT will help. RD: whether you have a pragmatic or a theoretical interest, you should be involved because the ZIG will have to vote on it at some point. MH: discussion goes on the ZIG list. SF: Oracle is interested, IBM, HP, Fujitsu, others; they are workng with Oracle on their object-relational DBMS in Australia. GROUP PROPOSAL (RD) Issues: RD: the Group proposal has two objectives: (1) to look ahead and do a search followed by a Sort or Present. (This was previously called the Concatenation proposal.) (2) to do things in one round trip without overloading the requests. For example, a Search followed by a Present could eliminate Piggybacking. Earlier proposals were derailed because people wanted to incorporate too much, e.g., scripting capabilities. RD is revisiting the issue because of the ZSTARTS initiative, proposing that the ZIG use Group with some qualifications in version 3. He specifically proposed using Group as an implementors agreement in version 3 -- Group without the Init. And balloting Group with Init included in version 4. Discussion: LW asked what's the problem with using Group with Init in version 3? RD: version 3 is rigid that an Init is required initially, not a Group. CL observed: RD has provided a good history of the thinking regarding the desire to do multiple Z39.50 operations in one package, to reduce network traffic and simplify. Perhaps this is a good place to say that we don't need another complicated feature in Z39.50. CL proposed making this a separate "proxy" protocol in front of Z39.50, which would make it version independent and interoperable with existing Z39.50 servers. The proxy could also do authentication. RL: the drawbacks are not minor: trigger-resource-control problems, concurrent operations become very difficult, everything has to be encapsulated in a Group. CL: agreed, but much depends on how you conceive of this operation. [Lost track here momentarily.] RL: if we adopt this Group proposal, then we will prune Piggyback out of the standard, and we'll be stuck with Group for everything. CL: certainly we've raised the possibility that we can prune Piggyback, if we accept Group, but the two are distinct issues. BW likes both RD's and CL's proposals; he wants a Group facility inside and outside of the protocol. LW likes the Group idea. The advantages of the proxy approach are really independent of the Group proposal. LW would support one or the other, not both. RD: is it possible to progress both of these? If so, it is urgent to have Group available soon; it will take longer to work out the details of proxy. CL: it would be awkward if a Group request came to the proxy. He thinks both paths would progress at the same pace in terms of implementation; there's no reason why specifications can't be moved along in parallel. It would be nice to keep similar data structures for the two (e.g., target IP addresses). KT suggested that the request have a bit that begins and ends the Group. RD: that approach would not allow us to use this in existing versions of the standard. MH was concerned that both proposals really open the door to doing many things that don't make sense. If we do either of these proposals, we give up a lot of the power of the state model. He understands the reasons for these proposals, but argues for a very restricted sense of the combinations allowed. This is not possible with CL's proposal, which isn't even necessarily restricted to just Z39.50 PDUs. RD: no one has objected to the absence of restrictions on what PDUs can be grouped. We should leave it up to the origin to not send silly sequences. DL: this is not a problem. Why invent a boogey man here? If the server doesn't support multiple Group responses, it indicates as such. In response to MH, DL did not see any interaction with state tables. The only interaction between Group and state tables is what you do with failure, which is throw away that stuff. RD: by handling the whole thing as a Group,there is no intermediate state. MH: Hmmm. JK: RD's wrapper could be the wrapper for the 211 port. CL: by definition, you can't have a state table problem with the proxy approach because there is no state table. LW: if we take the approach that KT proposed, we can do it in v3, but how does this differ from concurrent operations? RD: concurrent operations doesn't solve the problem because there's no way for the server to know what request is coming next (no synchronization). With Group you know that the server got all the stuff at once. LW: then KT's approach could be done, which would dispense with the need to put a wrapper around a Group request. It seems that we can accomplish this without changing the ASN.1. Liv (?): concerned about the grouped Search, Sort and Present. Will the user want to pay to sort and present a large result set? Who decides what to do? RD: the smart server would fail the Group request. Liv: if you Group, you lose control. CL: it's the origin, and not the user, who's going to decide to use this. He sees this as useful in constrained circumstances, e.g., applications like the Web that don't support interactions, situations in which you aren't concerned about running up a bill. LW: in circumstances where the Group failed, can you get any of the results? RD: the whole group can succeed, the whole group can fail, or there can be partial success -- in which case results to that point are returned. LW: how do you handle it if Sort fails, since sort changes the result set -- in which case you may no longer have a valid result set. [Here DL, BM, MH and others bickerd about whether or not state tables are affected and the execution of grouped PDUs.] RL: a Group is not identical to submitting requests in sequence. JZ: a failure situation is different when you do look-ahead than if you do things in sequence. CL: the problem is failing at the Group level -- you fail because of clairvoyance (the look ahead). MH asked for clarification: do we fail at the Group level or at a particular PDU? Stuart (?): the user should have the opportunity to decide whether or not to Group requests. [He made two other quick points, but I lost them.] KT agrees with LW that Init is required with Group. We should have an unconfirmed message that you send when you want to initiate a Group, then send the requests, then send a confirmed message that ends the Group. CJ doesn't see any reason why the server should not be able to fail because of clairvoyance. If grouping will eliminate Piggyback, what about the conditional semantics that are in Piggyback that are not in Group? MH: Good point. Discuss it over coffee. [break time] RD suggested that we put the bit in the Init (to begin the association) that refers to grouping as conceived within the Group proposal. The Init would not be within the Group PDU, but would logically be part of the Group, which would immediately follow and be encapsulated with the Init. This would solve the problem of migrating from one version to another. LW: if you set the bit in the Init and immediately follow with the Group, the Group is not negotiated, but assumed. If the target does not support Group, what happens? RL: a protocol error. RD: no. The bit does negotiate the Group. LW: this moves us to a single-request, single-response model; he prefers an Init request and response before sending the Group request, which would enable real negotiation. MD: we skipped the discussion of requirements. If you really want a single request and response, then you can't chain commands. JZ: CL's proposal meets the need and we don't have to deal with Init. Wayne (?): don't know that we need an Init response in all cases because many clients work with known servers. He's willing to let the client gamble. The proxy server does not address the need to group. SF wants to be able to do Group after Init all in one move. RD: nothing suggested precludes Init negotiation prior to sending a Group request. If you include the Group bit in Init, you may encapuslate Group with Init or send the Group request subsequently. RL: in the absence of ASN.1, he presumes that the encapsulation method would be OtherInfo in Init. That's a dangerous precedent -- that could be done with any other PDU. Wayne: We can explicitly bound the behavior we're allowing. Or we could concatenate rather than encapsulate. BW: you can always ignore OtherInfo. If the Group request was in OtherInfo, I would ignore it. Outcome: RD: the need is urgent. Can we agree or do we need to drag this out? BW: isn't this discussion irrelevant to ZSTARTS, where the primary need is a look-ahead Sort and single round trip stateless interaction? CL was not sure how important the need for look-ahead really is. He thinks they are really more interested in the single transit transaction. RD could not distinguish what the Web (ZSTARTS) folks really need from what they want. SF: to succeed with ZSTARTS you need the single round trip transaction. ZIG needs to reach concensus on how to do this. (?): how about two bits in the Init -- one to negotiate Group, and one to say here it is? RD: that would work, but we can't craft this right now (we're running out of time at the meeting). He wants to set a time frame to end this discussion in the next month. DL proposed: to address ZSTARTS, make Group an external and tell them we'll use it in OtherInfo. If you're not going to recognize the external in OtherInfo, you're not going to do Group anyway. (?) This does not require concensus from the ZIG. KT likes the two bits proposal. Wayne reiterated the isadvantages of OtherInfo. We could put new field in Init or use concatenation. RD, RL: a new field would require a new version. And there would be erver problems.... MH: we need to end this discussion; move it to the list. RL agreed that the safest thing to do is to move to the list. We have multiple sets of requirements, but most of the ZSTARTS needs can be solved with encapsulation in the Init. Other issues should be tackled separately. ASN.1 1990 (RD) Issues: RD: Pat Harris of NISO raised the issue. There is serious discussion about withdrawing ASN.1 1990 version in favor of the 1994 version. The ZIG has a critical stake in this. De-certification of ASN.1 1990 would have serious implications for Z39.50 applications. The ZIG discussed the new version in the past, but it hadn't been approved yet and was not adopted; discussion did not resume after the new version was approved. Is it even a remote possibility that we can begin working on ASN.1 1994? Version 1990 is not a subset of 1994. There are clear incompatibilities, not just extensions. Discussion: Wayne: some things in 1994 deprecate things in 1990. There are some things in 1994 that would solve problems for us (character strings?), but it will take some study. RD: we need to formulate a plan. We're probably going to say that destabilizing us at this time would be a serious threat. We need people to analye the 1994 standard. Implementors will need tools (compilers, etc). We need to consider adopting ASN.1 1994 early next year or in version 4. MD and RD briefly debated the ZIG's implementation of ASN.1 1990 and whether they were in compliance with the standard. Whatever the case, we cannot rewrite Z39.50 version 3 in ASN.1 1994 and be compatible. CL was trying to understand the interoperability implications of converting to ASN.1 1994. Is it likely that the conversion will not be backwards compatible with 1990 versions? JZ, DL and Wayne: aside from "Any" and some other things, it's compatible. Why consider migrating to a new version? ASN.1 is dead meat. If we're going to complicate our lives, why not use modern technology rather than 1984 technology just for minor improvements in ASN.1 1994. RL: are you suggesting that we drop the proposal to migrate to 1994 or consider leap to something else? DL: yes. LW: 1994 deprecates general string. What affect does this have on our applications? JZ, MD: you can legally escape out of general string to use ISO646. RL is worried about efforts in ASN.1 1994 to disentangle transport and abstract (?). Conversion to ASN.1 1994 will renumber all the tags. RD: DL's radical suggestion merits consideration. We're at a critical turning point. One possibility is to keep 1990 as long as it's practical, then switch to something else entirely. BM: in the time frame we have, we must work with ASN.1 1990. Given the lack of tools, we cannot migrate to ASN.1 1994 and we cannot tolerate having ASN.1 1990 deprecated or jeopardized. RD: the ZIG concensus is that we must make a strong statement to NISO to this effect. DL explained that he was not proposing a new technology; there is existing technology ("IDL and other stuff") that has more transport independence than what the ZIG uses. Manufacturers are discontinuing ASN.1 tools. Where will we get the tools? Outcome: MH concluded: we'll tell NISO to leave our ASN.1 alone. If they refuse, we won't talk to them anymore and we'll have to go out and find new tools. MD: the current standard needs some extension to the comments about international string and character sets. We'll take the discussion to the ZIG list. SIGLA MEETING (RL) North American OIW RL explained that the meeting was "nearly proforma." There were three agenda items: (1) rubber stamping of the current GILS profile, (2) agreement to turn maintenance of the GILS profile over to the new SIG for GILS, and (3) ratifying the ILL profile proposals. Questions? Approved. Meeting over. The OIW is dying. NIST has withdrawn interest and support. IEEE is in effect hosting OIW, but they don't have any legal standing. (There were laws that enabled vendors to get together and make agreements about things to be sold to the government.) It looks like the OIW will turn into a ZIG-like organization with all volunteer work. Within 6 months we should see official withdrawal of government support. SIG GILS is a GILS group organized in Washington D.C. They significantly broadened their area of interest beyond federal GILS. States are interested and will probably adopt GILS faster than the federal government. We forced some international awareness on them as well, recognized in their charter. RL and (?) approved formation of the new SIG for GILS. Their work is not related to the protocol per se, but focuses on the profile. When there are implications for the protocol, they'll be brought to the ZIG. PROFILE FOR ACCESS TO DIGITAL COLLECTIONS (RD) PROFILE FOR ACCESS TO DIGITAL LIBRARY OBJECTS (RD) RD: in the interest of time, RD withdrew discussion of the digital collections and digital library objects profiles. BW asked a question about profiles: where do we draw the line on profiles between what we would like and what the protocol will support? Since we don't negotiate profiles, and we don't even know what profile we're dealing with, .... RD: the principles intended to guide profile development are based on a system that need not negotiate profiles or have a mechanism to specify the profile in use. But we haven't achieved interoperability among profiles. There should probably be guidelines for profile development, but we haven't done that either. LW: are we approaching a situation where we do need to negotiate profiles? The profiles make very strong statements about what will or will not be used. Operating with no knowledge of the profile creates problems. We need conditional statements based on the profile in use. MH will make negotiation of profiles an agenda item for a future ZIG meeting. RD said that negotiating profiles was supposed to be on the agenda at this meeting (MH forgot), but RD abandoned his topics, so.... MH: the issues require more time than we have now. Take the discussion to the ZIG list. DEFECT REPORTS (RD) Nothing prepared. Dropped. PENDING INTERPRETATIONS AND CLARIFICATIONS (RD) PENDING EXTENSIONS (RD) (See handout.) RD: are there any points to be discussed? Questions about truncation (BW) and semantics of CCL in BIB-1 (?). KT: change "client" to "origin" in Extensions pending to Item Order Attribute Set. BW asked a question about the proposed use attributes for Item Order (Extended Service). He normalizes customer IDs (e.g., social security numbers). Do other people have problems normalizing IDs? MH sees this as one specific incident of a broader problem that the ZIG has not addressed. SD: the problem may be vocabulary, i.e., "customer purchase order number." Give the attributes the same names as the ASN.1. The ZIG agreed. LW asked a question about Scan "occurrences." Is it the number of total occurrences of the term, or the number of records where the term occurs? RD: the number of records where the term occurs. BW asked a question about pending extensions to the BIB-1 diagnostic set and suggested taking out the reference to Init. RD: OK. (?): How are diagnostics returned on the Init? [I lost the response.] BW asked a question about unit definitions. He wants some specific units to be added to the Z39.50 standard (a) if they are not covered by other standards or (b) if they create interoperability problems if they're not clarified in the standard. BW proposed the unit of "seconds" be added to the Z39.50 standard. LW will provide access to the STN international units standard documents. RD will create a table in the standard that defines (names) needed units and references relevant standards for other units. MEETING FREQUENCY AND SCHEDULING LW proposed that we meet twice a year for 2+ days. JZ said that we need more time. SF recommended twice a year, but more time, perhaps with a half-day break in the middle. RL was concerned that infrequent meetings means that lots of the work is being done by quiet subgroups meeting in Washington DC with results brought to the ZIG for rubber stamping. The next ZIG meetings were scheduled to enable three meetings in 1997. April 7-9, 1997 - Washington D.C. The rooms at the Library of Congress are reserved for three full days. (But the sixth Web conference may also be at that time. Is that a problem?) August 19-22, 1997 - Copenhagen Denmark Library Center (tutorial and ZIG meeting) MD will coordinate the tutorial. Denise A. Troll Acting Assistant University Librarian Library Information Technology Carnegie Mellon University Libraries 4825 Frew Street Pittsburgh, PA 15213 troll+@andrew.cmu.edu 412-268-8599 (phone) 412-268-6944 (fax)