REVIEWER GUIDELINES
REVIEWING PAPERS: WHAT TO LOOK FOR AS A REVIEWER
GUIDELINES ON USE OF GENERATIVE AI
FINAL DECISIONS: AFTER THE MANUSCRIPT LEAVES YOUR HANDS
The IEEE Open Journal of Control Systems (OJ-CSYS) is a new publication of the IEEE Control Systems Society covering the theory, design, optimization, and applications of Dynamic Systems and Controls. The journal’s main mission is the promotion of open access to all control systems research and education publications, including software, and data by drawing on the knowledge and help of the expert technical community. To that end, Reviewers play an integral role in the success of the journal by evaluating in a timely manner both the quality of the technical contribution and the quality of the assigned papers.
Reviewers are picked for being leading researchers in the field, likely having authored relevant papers that have been previously presented in high-profile journals and conferences. In addition to this criteria, our goal is in selecting those for whom the paper is on a topic that they would really like to see or to say something about. Having Reviewers who are interested in going over the manuscript is likely to yield a positive experience for the Reviewer and a succinct yet sufficiently detailed evaluations and recommendations for the authors. Please further note Reviewers should not be at the same institution as the authors of the paper.
PAPERPLAZA
All aspects of the manuscript review process are handled through an online submission system called PaperPlaza. To act as a Reviewer, you will have to be set up with the system/create a PIN for yourself:
1. Visit css.paperplaza.net/conferences/scripts/start.pl
2. Select “PIN” at the top left-hand menu
3. Follow the instructions to “Register a new PIN”
4. Enter your relevant information, including updating the keyword database
*Please note that a new PIN does not need to be created if any personal information changes, such as affiliation. Instead, please modify your personal information using the last login. For those who have volunteered with the journal, please ensure that someone from the editorial board has added you to their own list of potential Reviewers through the system in addition to setting your PIN up. This means the Associate Editor will be able to find you through the system when he or she is ready to request a review. Upon selection, the request will be forwarded to the email listed on your PaperPlaza profile.
DELEGATING REVIEWS
OJ-CSYS permits “delegating” reviews to a qualified doctoral student or junior researcher. Once the junior reviewer completes and saves the review, the original reviewer then must effectively endorse it, and also has the possibility of improving it further before the review is submitted. The review credit goes to the reviewer to whom the review was delegated.
To delegate a review, first accept to review the paper, then click on the link “Delegate.” The Delegate link remains available until the review is finalized and submitted by the reviewer originally requested to review the manuscript.
Note that the delegate reviewer can only “Save” the review for the primary reviewer’s approval, the primary reviewer must “Submit” the review before it becomes available for use by the Associate Editor handling the manuscript.
REVIEWING PAPERS: WHAT TO LOOK FOR AS A REVIEWER
In general, there are things that will be essential to the success of each manuscript. As in a literature review, the authors’ depth of knowledge and recognized expertise in the proposed topic, for example, as well as the importance and timeliness of topics, technical correctness clarity, citation of enough references, inclusion of enough examples, and the coverage of basic concepts are all things you will be looking for. Negative comments or opinions should always be discouraged.
In addition to these common factors of success for all manuscripts, certain focus should be given to a manuscript depending on its article type. There are three different article types for the OJ-CSYS, each with a different aim to keep in mind as you do your evaluation.
1. Regular PapersRegular papers are standard journal articles presenting significant research on analysis relevant for Dynamic Systems and Controls, and/or applications.
Some questions to ask are:
- Does the research focus on new, previously unpublished, and interesting contributions to the field of Dynamic Systems and Controls?
- Does the manuscript have a technical emphasis?
- Is the research relevant to answering a fundamental question of the field?
- Does the research focus the advancement of some theoretical aspect or of the application of Dynamic Systems and Controls?
Regarding applications, contributions can range from experimental results using a new method for a given application, to new concepts for real-time control architectures, to novel modeling techniques, or to innovative sensors. Descriptions of new application domains where control-engineering methods have not been applied previously are also acceptable contributions.
Technical correctness of the paper results, of course, is mandatory but does not necessarily ensure the paper acceptance. Being able to solve a problem with correct mathematics does not necessarily imply that the problem is worth solving.
2. Overview/Position PapersOverview/Position papers present a summary of a research area in the field of Dynamic Systems and Controls. They are mostly survey papers, but with tutorial-like elements.
Some questions to ask are:
- Does the paper provide a good introduction to the topic being surveyed, covering a wide breadth of that topic?
- Is the paper accessible to a broad audience?
- Does the paper provide key definitions and statements that allow a reader get a clear grasp of the fundamental components of the area being reviewed?
- Does the author address main questions in the area?
- Does the paper cover the significant contributions to the field?
- Does the manuscript cover an overview, as well as pros and cons of important competing techniques? (Overview Papers can make more emphasis on one particular approach to solve a set of questions, but cannot fail to cite competing approaches, to provide the reader with a comprehensive view.
- Does the application yield experimental results using a novel methodology and/or algorithms?
- Is the application a new concept demonstrating real-time control architectures?
- Does the application have a novel modeling technique or employ innovative sensors?
- Does the manuscript describe a new application domain where control-engineering methods have not previously been applied?
These are short papers presenting future challenges and new developments in Dynamic Systems and Controls. There are two types of papers: Perspective or Technical.
Perspective position papers provide a personal viewpoint on problems related to an emerging research area in Dynamic Systems and Controls. Perspective papers can either:
(i) Outline key challenges, provide guidelines, and future agendas for the field, and identify collaboration opportunities for researchers in Dynamic Systems and Controls with other areas;
(ii) Contextualize findings, adding a new dimension to the field, formulate questions that are related to the ethical, philosophical or legal dimension of a control systems technology, its public’s perception/acceptance, broad policy adoption, and social implications.
For Perspective papers, some questions to ask are:
- Is the paper written in a non-overly technical way, so as to be widely accessible to the broad scientific community?
- Are solutions or recommendations supported with arguments and evidence?
These papers can include some speculative and forward-looking content to stimulate new approaches and debate.
Alternatively, Technical position papers identify technical bottlenecks in current research, its open problems, and new techniques to solve them.
For Technical papers, some questions to ask are:
- Does it provide an account of available tools?
- Does it alert the audience to the importance and/or urgency of a problem?
- Does it provide convincing arguments of why the new identified techniques are promising to advance the state of the art?
The style of these papers can be more technical, but still accessible to the Dynamic Systems and Controls community.
4. Tools PapersUnder this category of papers, authors can submit i.) tutorial-like papers describing new testbeds, software, data, and benchmark tests with appropriate links to these tools, and ii.) report on the performance of control algorithms, as compared to others on benchmarking tests, sophisticated simulation (high-fidelity, with hardware in the loop) or other case studies with practical relevance.
For the first set of papers, some questions to ask are:
- Is the paper written in a tutorial-like way?
- Does the paper describe a new, accessible testbed, software, benchmarking test, or data?
- For a testbed, does it provide a clear description of the elements, how it is built, how the testbed elements are integrated, what the requirements to access the testbed are, and what the capabilities of the testbed currently are, how to interface with the testbed, and usage support.
- For software, is a comparison with similar software made? And are arguments for the adoption of such software should be provided?
- For data, it can be synthetic or real data. If synthetic, does the paper describe how it was created in detail? Does the paper make the case why this data is good enough to perform a given experiment? If real, is the data completely anonymized? Does the paper describe how the data format? Does the paper describe how the data can be imported, and managed?
- Is the corresponding supplementary material easily accessible? Are you able to download and access the repository with the software? For data, a reviewer should verify that the data is accessible, downloadable, and can be reasonably understood by a reader.
- Can the software be easily understood by a reader?
- Does the corresponding supplementary material work? Are you able to run the demo programs? (An evaluation of the quality of each line of code is not necessary, but reviewers should inspect the code to check if its main structure and components are clearly described with comments within the code.)
For the second set of papers, some questions to ask are:
- Is the research case study of a significant nature?
- Is it bringing the application of a control methodology closer to solving real-world problems?
- Does the paper compare the benefits of a given methodology against an accepted baseline?
Assuming the manuscript has passed the test related to each article type, the Reviewer can go on to evaluate whether the presentation is clear, concise, and complete. In other words, is the paper readable? Are all concepts clearly explained? What sections were hard to follow? Is all notation and terminology clearly defined before it is used? Are there places where more detailed explanations are needed? Are there parts of the paper that can be made more concise or technical arguments that can be shortened? Is the importance of the problem sufficiently stressed? Is the paper written in proper English?
The key to a good evaluation of the presentation is to be as specific as possible. Your review will guide the authors in making revisions to the paper. You need not re-write the paper for the authors, but you should tell them as precisely as possible where the revisions need to be made and why.
The role of reviewers is to both evaluate the content of the submitted papers and to help authors improve the quality of their manuscripts. Suggestions to improve the paper or to correct possible flaws should be presented in a clear and polite manner, and always be constructive in their criticism. This is necessary to convey a respectful tone, carry out a constructive discussion, and avoid misinterpretations that can lead to unnecessary, negative emotional responses.
When making recommendations, you should provide enough detail – examples, evidence of your argument and/or specific citations – for the author to understand why you are making the recommendation. Even if you believe a manuscript to be seriously flawed, try to provide suggestions for how it might be improved. Finally, Reviewer comments and conclusions should be objective and free from personal or professional biases.
GUIDELINES ON THE USE OF GENERATIVE AI
Generative Artificial Intelligence (GenAI) tools include Large Language Models (LLMs) such as Open AI’s ChatGPT, Google’s Bard, and Anthropic’s Claude. Reviewers may use GenAI to improve the exposition of their reviews, but they are ultimately responsible for their final review and any statements it contains.
To maintain the confidentiality of the review process, because GenAI tools might store and use input submitted to augment the data sets they use for training purposes, reviewers and editors shall not upload into a GenAI tool any part of a paper they are evaluating. For the same reason, reviewers and editors shall not upload into a GenAI tool any part of their review/report that contains identifying information about the paper or the authors.
For the most up-to-date information on policies surrounding the use of GenAI, please refer to the IEEE CSS guidelines here.
Submission of a manuscript signifies that it has neither been copyrighted nor published, submitted, or accepted for publication elsewhere. IEEE policy requires that authors, when using their own previously published or submitted material as a basis for a new submission, must disclose such use in their cover message and “cite the previous work(s) and very clearly indicate how the new submission differs from the previously published work(s).” [IEEE Publications Operations Manual, Section 8.2.4F.] The submission of the previously published material as Supplementary Material is also required.
If a submitted manuscript has been published or has been accepted for publication in the Proceedings of an IEEE conference, it may be considered for publication if evidence is provided that it adds value relative to its conference version (for example, it contains detailed proofs omitted from the conference version, new material, and/or additional numerical results). This ensures consistency with the policy stated in the previous paragraph.
If, at its time of submission, a manuscript has also been submitted for publication in the Proceedings of an IEEE conference, it will be considered for publication in the journal with the understanding that, should it be found publishable in both venues, evidence will be provided that its final version adds value relative to its conference version, as explained in the previous paragraph. Please refer to the CSS Policy for Overlap for further information.
In either of the above cases, the conference version must be uploaded as supplementary material.
OJ-CSYS accepts computer code associated with the final accepted manuscript. As Reviewer, you are asked to review this supplementary material, which will be submitted through PaperPlaza or a repository such as Code Ocean and IEEE DataPort, in which case a visit to the respective website will be necessary.
Instructions are included below to access the two main repositories, and in addition, authors will be asked to include a SUMMARY file describing the overall components and intent of the code, as well as a README file describing the steps required to reproduce simulated results and/or to build/execute the provided code.
OJ-CSYS accepts computer code associated with an article (e.g., implementing algorithms), which can be submitted with the final accepted manuscript, and users in IEEEXplore will be able to discover and access the link to run the algorithm in Code Ocean.
As reviewers, you will have to review this supplementary material. Please go through the following steps to run and review the code:
- Visit repository through CodeOcean.com/explore or IEEE Xplore Widget
- Sign Up with Code Ocean for free
- Search for capsule you want to review (by keyword, research field, title, author, DOI)
- Click “Reproducible Run” on top, right-hand corner.
Datasets can similarly be submitted with the manuscript. The authors will be asked to upload the data either into PaperPlaza along with their submission or into their own repositories (Dropbox, Google Drive, Github, or another repository of your choice) during the review process. IEEE Data Port will then be used once the article has passed the review process and has been chosen for publication.
The OJ-CSYS aims at ensuring rapid dissemination of the most recent results in the broad field of Systems and Control. The OJ-CSYS has a target of 20 weeks from submission to final decision. To provide a service with such high standards, the Editorial Board and the Reviewers involved in the review process must comply with tight and demanding constraints. As a Reviewer, you will have to observe the following strict deadlines:
- Four weeks (28 days) from reception of original manuscript to first review
- Two weeks (14 days) from reception of a revised manuscript to re-review
FINAL DECISIONS: AFTER THE MANUSCRIPT LEAVES YOUR HANDS
The manuscript review process is handled by an Associate Editor, who prepares a report and makes a recommendation on the basis of peer reviews. The final decision on publication, whether accepting or modifying this recommendation, is taken by the Senior Editor handling the paper and by the Editor-in-Chief. The review process is single-anonymous: Reviewers are never known to the Authors. Also, the names of the Associate Editors handling the review processes are not disclosed to the Authors, who will only interact with the Senior Editors assigned to the papers.
Your contribution to the IEEE Open Journal of Control Systems is an invaluable resource for the journal’s mission of the promotion of open access to all control systems research and education publications, including software and data. Your feedback is therefore highly valued, and you can feel free to contact the Editorial Board with any questions or suggestions, especially concerning the review procedures. You may contact the Associate Editor who sent you the manuscript, the Editor in Chief or the Editorial Assistant.