Guidelines on Research Data and Reports

...an evolving document produced to foster good research practices

 
1. Responsibilities of Principal Investigators

Principal Investigators (PIs) have final responsibility for:

The validity and quality of the data and reports generated from their laboratory or clinical program.

Fulfilling the research, reporting, and publication standards, policies, and procedures of the Department of Neurological Surgery, the University of California, San Francisco, and the USPHS, National Institutes of Health, National Science Foundation, and Office of Research Integrity.

Formally orienting junior faculty, fellows, residents, and staff to those standards, policies, and procedures and, to the maximum extent possible, seeing to it that they are upheld.

Overseeing the work done by fellows, residents, and staff to assure that each has the knowledge, information, and skills necessary to meet the standards of the institution and scientific convention.

2. Research Data

2.1 Data Management and Review

In general, two primary forms of data records are maintained in a laboratory: the methodology notebook and the experimental notebook. Laboratories with several individual research projects also keep a laboratory master log. A data selection file containing data selected for publication and documents related to publication is kept for each paper resulting from a study. Standards for accurate collection and recording of data and for storing data are detailed in 3 of these guidelines.

2.1.1   The PI has final responsibility for:

The validity of the data.

Maintaining methodology notebooks and laboratory master logs relevant to the PI's laboratory and seeing to it that those books and all experimental notebooks, data selection files, and related data and records are kept and stored according to the standards set out in these guidelines.

Ensuring that data (a) are collected and recorded in the experimental notebook according to the standards described in these guidelines, and (b) are stored in a comprehensible way for others to have access to them.

2.1.2  In some laboratories, a staff research associate designated by and responsible to the PI may maintain methodology notebooks, oversee experimental notebooks, have laboratory management responsibilities, and/or instruct new fellows and residents in laboratory techniques and protocols. In those laboratories, the PI meets with the staff research associate to review research progress and data at least once each month (see 2.1.4).

2.1.3  The PI holds scientific meetings with junior investigators, fellows, and residents at least once each month, at which time the PI reviews the experimental notebooks and related data and records. For educational purposes, however, PIs are encouraged to meet with them one-to-one more frequently, at least once a week (see 2.1.4).

2.1.4  PIs are encouraged to initial and date the latest page of each experimental notebook reviewed in the event that documentation of these reviews is needed at a later date.

2.2 Statistical Design and Analysis

Investigators are encouraged to consult a statistician during the earliest stage when designing a study, and then again when interpreting statistical data.

2.3 Use of Pooled Data

An investigator wishing to base a study on pooled computerized data that was generated by anyone other than himself or herself alone must discuss the project with the Director of the BTRC and the PI who derived the data before work begins to assure proper authorship, acknowledgment, and attribution of ideas and data.

2.4 Ownership of Data

Methodology and experimental notebooks and related data and records are the property of the University of California. They may not be removed from the BTRC, although investigators may take a photocopy of all or part of them from the BTRC. [Note: When a PI resigns from the University, arrangements can generally be made to transfer ownership appropriately.]

2.5 Storage of Data

PIs store all data notebooks and related data and records in their laboratory for 5 years after the date when funding for a study ends. They may then continue to store them in their laboratory or may make arrangements with the BTRC research administrator to have them moved to the University's storage facility; both the PI and the BTRC research administrator keep a record of the information necessary to retrieve the materials from the facility. Data notebooks and related data and records for any study may not be destroyed.

3. Standards for Databooks

3.1 Data Notebooks

3.1.1 Bound Notebooks with consecutively numbered pages. These databooks, with a permanent (sewn) binding, are the hard copy of choice for data recording. Duplicate pages in the notebook are used only to generate a carbon copy or to paste in auxiliary material-not to record data.

3.1.2 Loose-Leaf Binders. These are used instead of bound data notebooks, at the PI's discretion, to log all or portions of experimental records or generated data. All pages of the loose-leaf binder should be numbered consecutively before the binder is used. Each page should be identifiable as consecutive and belonging to a specific experiment according to a system created and followed in the laboratory (e.g. an experiment-identification number followed by the sequential page number and the investigator's or technician's handwritten initials: 1.23.MT ). The experiment-identification system used is clearly described on the first page of every binder in order to permit proper archiving and retrieval of data. When an experiment is completed, the consecutive pages of data and notes may be inserted into a plastic sleeve(s) for permanent storage in a binder for the one experiment, or in one binder including all experiments for the study.

3.1.3 Data and relevant material that are stored separately (e.g. computerized data files, microscope slides): see 3.4.4-3.4.6.

3.2 Laboratory Master Log

For studies involving several investigators, or for laboratories with several individual research projects, the PI(s) keep a master log that serves to catalog the experiments of the whole study or laboratory. This central log, a hardbound databook with consecutively numbered pages, should contain:

the titles of the studies done by everyone in the laboratory,
the investigators' names,
the inclusive dates of the experiments, and
the location of the experimental notebook and any raw data, computer files, or other relevant materials stored separately for each logged experiment.

3.3 Methodology Notebook

3.3.1 In each laboratory, certain techniques or protocols are used in common on a daily basis, such as specific cell-culturing techniques (e.g. cell transfers, dilutions, cell counting, media preparation), irradiation techniques, tumor implantation procedures, neurologic examinations, animal anesthesia, electrophoresis procedures, and others. Specific details about each of these commonly used methodologies (including the statistical) are documented and numbered or assigned reference notations that facilitate citation in experimental notebooks. Such documentation serves to standardize all experiments that generate data of the same form and is also instrumental in training new laboratory personnel.

Notebooks have a section for each technique, and each section contains all versions of the technique, each dated for reference.

A 'table of contents' to the methodologies is kept at the front of the notebook.

The specific entries in the methodology notebook are modified as improvements in the procedures are developed. Changes are noted precisely and dated in the methodology notebook.

Outdated or discontinued methods remain in the methodology notebook, with a notation of the precise date the modified or new method(s) went into effect (e.g. so that earlier methods can be readily retrieved for reference in writing a research report).

3.3.2 The methodology notebook is the final and absolute arbitrating reference when questions of technique are raised in the context of the educational and training responsibilities of the BTRC.

3.4 Experimental Notebook

3.4.1 The experimental notebook is the vehicle by which the experiment is fully documented.

3.4.2 The first several pages of the notebook are reserved for a 'table of contents,' in which are listed, as the study develops, the experiments and the pages on which the data are located.

3.4.3 The following minimum information is entered for each experiment; PIs are encouraged to make up a 'boilerplate' page that can simply be filled in with this information.

Title of study
PI's name
Date the study starts; date it ends
Associate investigator's name(s) (i.e. staff research associate/postdoctoral fellow/resident/graduate student)
Brief statement of hypothesis or study goals
Cell line (passage no.)
Animal strain and supplier
Specific animal identification no. (large animals)
Source of analyzed material
Tumor type & passage no.
Drug type (lot no. and/or source), dose(s), dilution(s)
Radiation source and dose
Special reagents (e.g. antibodies, probes)
Cell culture batch/medium
Serum batch/medium
Experimental design
Study-specific treatment groups, projected number of subjects, and all other elements of the study design with reference to specific techniques or protocols from the methodology notebook [refer to each specific protocol or technique by its designation (e.g. number) in the methodology notebook].
Statistical methodology added to or deviating from that in the methodology notebook.
A 'time line' illustrating the sequence of study events (e.g. start--cells added--medium added--etc.)
Specific notes about special procedures or steps that differ from the techniques specified in the methodology notebook (e.g. changes in incubation time or temperature, concentration of trypsin, tumor cell inoculum, infusion rates, anesthetic procedure). Any variance from the routine procedures recorded in the methodology notebook should be noted in detail [refer to each specific routine protocol or technique by its designation (e.g. number) in the methodology notebook].
Raw data, or explicit instructions for locating the raw data or retrieving them from storage (see 3.4.4). Of particular importance are notations about excluded data or animals, with detailed information about why those data or animals were excluded.
A brief conclusion of the experiment, including a 'value judgment' about the validity of the experiment, whether the study needs to be repeated to validate it, what can and cannot be definitively concluded from the data, and other observations. Simple concluding descriptions such as 'bad study,' or 'data suspect,' are not acceptable. It is essential to document why the study or data were considered suspect. Justifications for positive judgments similarly should be recorded.

3.4.4 Raw Data. Whenever possible, raw data are stored together with the experimental notebook; e.g. they may be stapled on the duplicate page following the related databook entry or placed in a plastic sleeve(s) and inserted in the binder. Data too unwieldy to include are listed in the experimental notebook as they are collected, are described sufficiently for recognition, and are annotated with the name of investigator and explicitly where the data can be found (e.g. location of the tape or disc and its identification number). Raw data include, but are not limited to:

handwritten notes on e.g. cell or colony counts, tumor dimensions, physiologic endpoints, daily observations on animals or other visually measured data (e.g. NMR tapes) from which observations were made,
photographs, photomicrographs, negatives,
spectra, EEG recordings, evoked potential recordings
films, scans, images
slides (e.g. histologic sections)
dated hard copy of computerized data files (see 3.4.5).

3.4.5 Computerized Data Files. In the data notebook is included dated hard copy of these files or, if that is too unwieldy, dated summaries that describe the files sufficiently to find and recognize them, including the location of the data file and the particular computer disc(s) on which the data are stored.

3.4.6 Blinded, Cooperative, or Multicenter Studies. Data for blinded or double-blinded studies are kept in separate (perhaps smaller) bound notebooks by the respective investigators and are brought together with the experimental notebook(s) as a single unit for storage at the end of the study, when the code is broken. Data management for cooperative or multicenter studies is in keeping with these guidelines to the greatest extent possible.

3.4.7 Standards for Keeping Experimental Notebooks

3.4.7.1 Each entry in the experimental notebook should be able to stand alone, to permit others to replicate the work at any time, whether immediately or even years after it is made.

3.4.7.2 Experiments are logged in the notebook in chronological sequence.

3.4.7.3 Data are recorded chronologically as they are collected on consecutive pages of the experimental notebook.

3.4.7.4 Entries should be organized in such a way that someone not familiar with the specific experiment recorded can retrieve all the pertinent details of the study, from the hypothesis to the published article. Notes entered at the time of the experiment summarizing the goals, details, or problems can be invaluable during subsequent analysis or defense of the results and are therefore encouraged.Optimally, the experimental notebook is a journal of the study.

Databooks are kept only in ink and must contain no erasures or 'whited out' changes.

An entry made by mistake is deleted only by drawing a single line through it, preferably in ink of a different color. The deleted material should remain legible beneath the overstrike. Large blocks or a page to be disregarded are crossed over with an 'X' or diagonal line and marked e.g. 'OMIT'. The page should remain legible.

The corrected data are written beneath or beside the original entry. The explanation for the alteration is clearly written in close proximity to the alteration-preferably on the same page or on the facing page.

3.4.7.6 If any changes are ever made in the experimental notebook-including a change in values, correction of a mistake, or like alterations-it is absolutely required that those changes be dated and initialed by the person making the alteration, and a clear explanation noted as to why the alteration was made.

In permanently bound databooks containing duplicate (often perforated) numbered pages, only the original bound-in page is used to record data. The duplicate page is used only to make a carbon copy or to paste in auxiliary material, e.g. photographs; it is otherwise simply left blank.

Whether permanently bound or loose-leaf, only databooks with consecutively numbered pages should be used. Pages must never be torn from or replaced in a databook.

3.5 Data Selection File

Preparation of a research report involves the selection of specific experimental data from the experimental notebook. A data selection file, filed separately from the experimental notebook and clearly cross-referenced to it, is kept for each paper to be submitted for publication. The data selection file consists of:
 
Those data selected for reporting and their analyses (including e.g. graphic presentations and statistical manipulations). These are photocopied from the original experimental notebook and cross-referenced to that notebook page by page (unless the cross-reference is evident on the photocopy).
The rationale for selecting the specific data used in that particular paper, recorded narratively ("I selected this datum on the basis of X, I excluded this datum on the basis of Y."), including justification for the selection of specific data to make a curve or other statistical representation.
A document naming the coauthors and detailing their specific contribution(s) to the study.
A document naming the persons cited in the acknowledgments as contributing to the paper and detailing their specific contribution(s) to the study.
Any other material considered pertinent to selection of data, to authorship, or to any substantial related matter arising during the development of the paper.
At the completion of a research project, the data selection file for each paper developed from the project is archived in the PI's laboratory together with the experimental notebook(s) for the project and the photocopies of the relevant material from the methodology notebook(s). The manuscript development file is later added to the archive.

4. Authorship

4.1 The first author of a paper is named, coauthors are selected, and order of authorship is assigned and agreed on by all coauthors before a study begins. Although changes may take place, such an initial plan provides a context for the relative responsibilities and expectations of each investigator.

4.2 Authors should meet the standards described in the Uniform Requirements for Manuscripts Submitted to Biomedical Journals 1 [bracketed wording is summarized]:

All persons designated as authors should qualify for authorship, and all those who qualify should be listed.

Each author should have participated sufficiently in the work to take public responsibility for appropriate portions of the content.

One or more authors should take responsibility for the integrity of the work as a whole, from inception to published article.

Authorship credit should be based only on 1) substantial contributions to conception and design, or acquisition of data, or analysis and interpretation of data; 2) drafting the article or revising it critically for important intellectual content; and 3) final approval of the version to be published. Conditions 1, 2, and 3 must all be met.

Acquisition of funding, the collection of data, or general supervision of the research group, by themselves, do not justify authorship.

Authors should provide a description of what each contributed [and should expect that the journal may publish that information].

All others who contributed to the work who are not authors should be named in the Acknowledgments, and what they did should be described.

[When a group is named as authors of a multicenter trial, all members] should fully meet the above criteria for authorship. Group members who do not meet these criteria should be listed, with their permission, in the Acknowledgments or in an appendix.

The order of authorship on the byline should be a joint decision of the coauthors. Authors should be prepared to explain [to the journal editor the rationale for] the order in which authors are listed.

4.2.1 The senior author is the PI responsible for the validity of the data reported and may or may not be the first author of the paper. If the senior author is not the first author, his or her name appears last in order of authorship. The senior author is the 'corresponding author', i.e. is cited as the author to whom correspondence should be addressed.

4.2.2 The first author is the person who generates the data, collects and/or collates the data for review by coauthors, and writes the first draft of the paper. If these responsibilities are shared equally, those coauthors sharing decide the order of authorship; on the title page of the paper, their names are asterisked with a footnote stating that equal credit is due.

4.2.2.1 The first author is responsible for maintaining and archiving the data selection file (see 3.5).

4.2.2.2 If the first author leaves the BTRC before the data selection file is complete (and is not the senior author), responsibility for maintaining it reverts to the senior author.

4.2.3 The first and senior authors together decide whether or not another participant is to be a coauthor. Other coauthors of the paper are named on the byline in the order of the importance of their contribution to the research reported.

4.3 Order of authorship may be changed by agreement between the first author and the senior author; e.g. if any author recommends that changes of personnel over the course of a study call for revision of the initial authorship. All who have been named a coauthor should be notified of authorship changes before a paper is submitted for publication, and any changes should meet the authorship standards listed in 4.2.

4.4 Disputes about assignment or order of authorship that cannot be resolved among the authors should be referred to the Director of the BTRC.

5. Coauthorship

5.1 All coauthors meet at least once to discuss the selected data and the results of the work and to reach agreement on their conclusions, the general direction of the paper, and the work of others, if any, to be referenced or acknowledged.

5.1.1 This meeting is usually conducted in person but, if necessary, it may be by conference call or computer communications.

5.1.2 If these alternatives are unworkable, the senior author e-mails or otherwise sends the absent coauthor(s) a written summary of the coauthors' meeting. In such cases, the resolution of any disagreements on the work among coauthors should be documented in letters that are archived in the data selection file.

5.2 Coauthors should review the raw data on which their report is based.

6. Research Reports Submitted for Publication

6.1 Papers submitted for publication should meet the standards and requirements specified in the Uniform Requirements for Manuscripts Submitted to Biomedical Journals ,1 except when those standards conflict with the instructions of the journal to which the paper is being submitted.

6.2 Papers should include a notation of all funding for the work reported and disclosure of any actual or potentially perceived conflict of interest. To assure that all relevant sources of funding (e.g. grants, donor gifts) are acknowledged, consult the BTRC research administrator before submitting a paper.

6.3 All coauthors must read and approve the final draft of the paper before it is submitted for publication.

6.3.1 Before the initial submission and any substantially revised resubmission (see 6.3.4), the first or senior author sends a copy of the penultimate draft of the paper to each coauthor for final review. If differences still must be settled among coauthors the paper goes through another draft, which is also reviewed by all coauthors.

6.3.2 If a journal requires that all coauthor's signatures accompany the submitted paper, all coauthors must personally sign the letter or form (or a copy). Signatures in absentia are never legally acceptable. If an author becomes incompetent to sign or lost, note this in the signature space or a cover letter. One copy of the cover letter and form is included in the data selection file and one in the report's manuscript file.

6.3.3 When, after journal peer review, the revision of a paper involves additional or different data, a substantial revision of content or conclusions, a change in authorship, or a change in order of authorship, the renewed agreement of all coauthors is necessary and documentation of the renewed agreement should be kept in the data selection file. What constitutes "a substantial revision of conclusions" is at the PI's discretion.

7. Publication of Negative Results, Corrections, Errata, and Retractions

7.1 In addition to original reports of positive findings, the BTRC supports investigators in submitting for publication (a) papers reporting results that disprove or fail to replicate earlier published conclusions, (b) errata in published reports, (c) a correction of published work that is scientifically flawed, and (d) a retraction of any report determined invalid or fraudulent (consult 1).

7.2 BTRC investigators are responsible for informing all of their coauthors when a correction or retraction is warranted and for publishing it in the journal that published the paper originally. All coauthors of the original paper should coauthor a retraction together with any other investigator(s) later involved in the work. Conflicts and questions that cannot be resolved among authors are referred to the Director of the BTRC.

8. Questions of Research or Publication Practices Other than Authorship

8.1 BTRC personnel and trainees should refer questions or problems about research or publication practices to their immediate supervisor and/or PI.

8.2 BTRC personnel and trainees should refer any allegation of improper practice or misconduct in the performance or publication of research to their immediate supervisor and/or PI. The person alleging the claim must be able to support it with documentation. All BTRC personnel involved with such an allegation must observe due process during all phases of inquiry.

8.3 BTRC personnel and trainees should refer any question, problem, or allegation not resolved through the channels just described to the Director of the BTRC and thereafter, if necessary, to an appropriately responsible University officer.

 

Summary Checklist

...protocol on recording data and preparing for publication

___ Review relevant literature (ongoing)
___ Read original sources in their entirety
___ Photocopy references or log complete and accurate citations with inclusive pages

___ Plan experimental study design
___ Obtain statistical consultation
___ Ensure that all necessary research approvals have been obtained
___ Review and photocopy relevant sections from the appropriate methodology notebook for reference

In the experimental notebook, initially record:
___ Ttitle of study
___ PI's name
___ Date the study starts (inclusive dates after study ends)
___ Associate investigator (e.g. postdoctoral fellow, resident, staff research associate)
___ Brief statement of hypothesis or study goals
___ Cell line (passage no.); animal strain and supplier, specific animal i.d. no. large animals); source of analyzed material; tumor type & passage no.
___ Drug type (lot no. and/or source) and dose(s); radiation source and dose; special reagents (e.g. antibodies, probes); cell culture batch/medium; serum batch/medium
___ Experimental design in detail (e.g. treatment groups, projected number of subjects), correlated specifically with the methodology notebook.

Perform study, recording:
___ Specific notes about special procedures or steps that differ from specific techniques (including statistical) taken from the methodology notebook; record of any variance from usual BTRC procedures in specific correlation with the methodology notebook.
___ Raw data, including handwritten notes on data collection, endpoints, daily observations, location of photomicrographs and negatives, and particularly notations about excluded data or animals, with detailed information on why they were excluded.
___ A 'time line' illustrating the sequence of study events (e.g. start--cells added--medium added--etc.)
___ Brief conclusion, including a 'value judgment' on the validity of the experiment, whether the study needs to be repeated to validate it, what can and cannot be definitively concluded from the data, and other observations.

___ Analyze data

___ Meet with all coauthors at least once to discuss the raw and selected data and the results of the work, to agree on conclusions, to decide the general direction of the paper, and to agree on the work to be referenced in the paper.
___ Create a data selection file in consultation with coauthors as much as possible.
___ Select data from the experimental notebook and their analysis, including graphic presentations and statistical manipulations, and archive them in the data selection file, which is clearly cross-referenced to the experimental notebook.
___ Select pertinent references
___ Select journal most relevant to the work

___ Write the paper for the journal chosen
___ Send a working draft to coauthors
___ Revise paper until final revisions are complete
___ Complete and archive documentation in the data selection file (e.g. statement and order of authorship, contributors acknowledged, permission to use 'personal communications')

___ Submit paper to the journal

 

Reference

1. International Committee of Medical Journal Editors. Uniform Requirements for Manuscripts Submitted to Biomedical Journals. Ann.Intern Med 1997;126:36-47; JAMA 1997;277:927-34; <www.icmje.org>.
A brief guide to the particulars of manuscript preparation in a style that is acceptable (for review purposes) to nearly 500 biomedical journals. Does not supersede the Instructions to Contributors of individual journals.

 


Developed and written in 1989 by an Advisory Committee consisting of Susan Eastwood ELS(D) (chair), Philip H Cogen MD PhD, John R Fike PhD, and Harold Rosegay PhD MD, with Michael Berens PhD. Adopted in consultation with BTRC Director (1968-1997) Charles B. Wilson MD, Associate Director Dennis F. Deen PhD, and the principal investigators of the BTRC. Revised in 1995. Revised and updated in 2000 by Eastwood, Fike, and Rosegay in consultation with BTRC Director Mitchel S. Berger MD and Associate Director Dennis F. Deen PhD; updated in 2001 by Eastwood.


Reprinted in: Panel on Scientific Responsibility and the Conduct of Research, Committee on Science, Engineering, and Public Policy-National Academy of Sciences, National Academy of Engineering, Institute of Medicine: Responsible Science: Ensuring the Integrity of the Research Process (vol II). Washington DC: National Academy Press, 1993. Cited in: Korenman SG, Shipp AC: Teaching the Responsible Conduct of Research Through a Case Study Approach: A Handbook for Instructors. Washington, DC: Association of American Medical Colleges, 1994. Year 2000 revision reprinted in Bulger RE, Heitman E, Reiser SJ (eds): The Ethical Dimensions of the Biological Sciences (second edition). New York: Cambridge University Press, forthcoming.


The BTRC's Guidelines on Research Data & Reports is a working document produced to foster good research practice. Suggestions for additions or revisions are welcome.


Copyright 1989, 1995, 2000, 2001 by the Brain Tumor Research Center, University of California, San Francisco. Send suggestions, requests for copies, or requests for permission to reproduce these guidelines in whole or in part to the Brain Tumor Research Center, c/o Department of Neurological Surgery Division of Publications & Grants, University of California, San Francisco, PO Box 8099, Emeryville, CA 94662 [tel/fax: 510.594.2248; e-mail].

August 2001