What skills do you think are necessary to effectively use generative AI for research software development tasks?

Question details

Question code: genAI6

Detected type: Long free text

Answer order: not applicable

Number of answers: 132

Free-text answers

Free-text categorization is manually performed by the FutuRSI team and may contain errors. This is an ongoing work in progress.

These externally categorized source totals may cover a broader dataset than the selected country and submission filters. Percentages use the number of respondents who answered this question in the selected dataset; respondents may contribute to multiple categories, so percentages can sum to more than 100%.

Show 132 raw free-text answers (130 distinct)

code review × 2
prompt engineering × 2
A cirtical mind; good books or teachers in software architecture and software patterns; good books on robust software
A critical mind set and patience.
AI Competencies
AI is currently very useful to create algorithms, datastructures and boilerplate code. It becomes more important to be vigilant for misconceptions that the AI makes, taking more of a orchestrative approach. That said, until AI can take over coding completely, we must not outsource all coding tasks to AI, because the formalism of code helps us programmers to take important decisions and discover misconceptions. Generally the job shifts to doing more design work and less coding. I use GitHub copilot to write boilerplate code quickly and use chat to do reaearch on new areas and discuss ideas.
AI-literacy, critical thinking, practice
Ability to ask AI the right questions
Ability to evaluate output of LLMs, which, ironically, reduces to being basically equivalent to my entire skillset
Actually still understanding the code, and being able to reason with it
All SE skills are necessary because you have to be able to assess the information that is generated and trim it down, i.e. only take the code that is really necessary.
Avoid it.
Basic dev skills
Be able to understand the suggestions before applying them
Be an experienced software engineer, know what you're doing even without the AI, be able to thoroughly review the probably crappy but very plausible AI output, be able to treat AI like a very junior dev
Be aware that AI is not smart, is not able to think for you, and must not be used to directly generate critical parts of software or documents
Be very aware of the things that get generated and double-check everything and test rigorously.
Being thorough and sceptical to accept any answer
Building MCP tools, Building custom GPTs
Code Reviewing, Sehr gute Programmierkenntnisse
Code comprehension and validation, debugging, holistic understanding of the software project, project scope awareness, experience with generative AI tools and their failure modes
Code lesen und eigenständig bewerten - was sich leider durch massive Nutzung von KI schwerer (weniger oft) lernen lässt.
Code review, language expertise, pentesting
Code understanding, problem solving, problem formulation
Coding, Debugging, mastering version control system
Comprehensive understanding of everything AI produced; multi-language experience; review and text creation experience
Correct prompting
Critical Thinking/Judgement
Critical thinking
Critical thinking and understanding of the code
Critical thinking, testing AI generated output
Critical thinking. Not everything that generative AI spits out makes sense. Some knowledge of the work is needed to determine if the result is viable or not. Personally not a fan of generative AI and avoid using it as much as I can.
Crittically questioning the output of AI
Curiosity, clear problem definition and cautious use of tools (to avoid hallucinations)
Das wichtigste ist zu wissen wann man die KI nicht einsetzt.
Debugging, Software Design, Clean Code
Design thinking, architectural thinking
Die Fähigkeiten sind die selben wie für das effiziente entwickeln von Software generell. Man braucht vor allem ein solides Verständnis im Requirements Engineering und muss diese Vermitteln können (egal ob einem Bot, einem Mitarbeiter oder sich für sich selbst zur Strukturierung der Entwicklung).
Domain Knowledge, Critical thinking
Durchblick,Ambition,Verständnis,Programmieren
Einschätzung und Prüfung des Code
Erfahrung, kritisches Denken
Ethische Klarheit um KI abzulehnen
Fachwissen, Coding Competence
Formal software engineering training, understanding of domain research
General Concepts of Software Engineering, Software Architecture
Gute promts schreiben
I am not sure
In general, you have to already have some development experience / know what you're doing to be able to use these tools effectively. Otherwise you won't be able to spot mistakes or critically review their outputs.
Knowing its limitations
Knowledge
Knowledge of connected services, technical & organisational constraints, domain knowledge; AI needs to be guided for basicially everything except for code generation
Konzeptionelles Verständnis von diversen Techniken und Tools, die einem die KI ausspuckt, um bewerten zu können, was gut ist und was nicht. Das Zusammenfassen und Kürzen exzessiver, wortreicher KI-Ausgaben bevor man diese an Kolleg:innen weitergibt oder seiner Dokumentation hinzufügt.
Kritisches Hinterfragen, Belohnungsaufschub und Bereitschaft zu Lernen obwohl 'die Antwort' schon da ist (delayed gratification).
Lernen die richtigen Fragen zu stellen? Systemverständnis
Lesen, Aufmerksamkeitsspanne, Review, Architekturverständnis
Minimal understanding of software development workflows, basic programming knowledge
Need to be able to break down a task into subtasks that are comprehensible to the AI and testable by you, need to know enough to understand what is generated line by line and to check if what is generated is both producing the correct result and taking a sensible approach
Offenheit gegenüber KI, Zugriff auf qualitativ Hochwertige LLMs
One needs to understand what AI does and where it is going to struggle (and what it doesn't do). Where it struggles it can introduce errors but dress them up very convincingly.
Overview on available tools, Knowing when to use the tools; Critcal assessment of the results
Own expertise in software development
Previous coding experience
Programming experience to find flaws in the AI's output, understanding for architectures, appreciation of (automated) code testing
Programming knowledge
Prompting
Promting, Skepsis,
Proper human testing
Proper understanding of problem
Reviewing skills
Selber zu denken
Self scepticism
Senior RSE skills
Sich mit der inhärenten Unzuverlässigkeit von Sprachmodellen kritisch auseinanderzusetzen und einzusehen, dass sie zur Code-Generierung untauglich sind.
Strong foundational skills and knowledge, owning and taking responsibility for produced software
That's really hard - we are only just learning about it., If used well by experts then I believe it is a powerful tool., If used blindly then it is full of dangers.
Training of junior staff in the seemingly outdated modes of doing things. The model can only enhance what’s there
Training, choose the right AI, check and debug final code
Understanding how to check correctness of its code completion suggestions. In my experience its first answer is right 70-80% of the time.
Understanding the basics of gen AI, Understanding its limitations, Using it only as a basis always recheck the generated output and refine manually
Understanding the objectives of the development.
Understanding the scope of the software. Communicate to the AI what the users need. Understand the needs of the users and how they would like to benefit from the sofware.
Understanding what is right and what is wrong, handle bad code from non programming people, handle backslash if people understand the limits of ai and starting to avoid it
Using stuff efficiently while staying totally independent from bigtech
Verstehen des erzeugten Codes, um diesen bewerten zu können und Logikfehler zu finden
Vorwissen um die Ergebnisse beurteilen zu können, Prompting-Fähigkeiten
You have to be able to understand the code it generates and therefore be able to write it yourself. Only if you still understand your code it is an actual productivity increase. Better might be to not use AI to generate Code but to ask How a problem could be solved and discuss as form of rubber ducky debugging
a general understanding of the programming and not blind copy + execution of the code
bessere Konzepte für Testing
ciritical thinking, patience
code review, code review, code review, time management, data privacy, resilience against hypes
code review, scepticism, understanding of llm capabilities, software development
critical thinking
critical thinking and fact checking, solid technical foundations (including staying up to date), using AI as a sparring partner rather than as someone doing the work
critical thinking and reviewing! basic technical knowlegde about concepts and structures in software!
critical thinking, no over-reliance, having time to do things properly, research skills like investigating different solutions and weighing pros and cons
critical thinking, problem solving, judgment
criticism, tests
debugging/code review, testing, project planning
deep theoretical understanding of ML/AI, restraint, critical thinking
domain knowlege, programming skills, programming language knowledge
explaining well all important details, knowing how to configure with AI agents, keeping in mind privacy considerations
firm grasp of the programming language and its concepts, otherwise you aren't able to review the generated code.
generating permanent facts/skills to be reused between conversations, local inference (ideally on device but smth similar to GWDG would be okay too)
good software dev skills to double check/guide Ai
how to phrase queries to get the answer you are looking for immediately
humility
knowing how to protect source from GenAI slop
knowledge of how they work, programming knowledge for purpose of verification, how to use them effectively
knowledge of software
peer-review experience. Doubt every paragraph.
read software, evaluate software quality
sceptisism, targeted use
self-esteem, discipline
skepticism,patience,active knowlede
software architecture, low-level understanding of computer architecture
software architecture, programming
software architecture, testing techniques, ai agent management
software development, requirement analysis, basic understanding of research field the software is made for, data management
software engineering (skilled/advanced),programming (skilled/advanced)
testing, integration, licensing, dependency management
the ability to actually understand and test what generative models produce
tieferes Verständnis des Gesamtcodes, um im prompt alle notwendigen BEdingungen exakt definieren zu können und dann den generierten Code hinsichtlich der Wechselwirkung mit anderen Teilen einschätzen zu können
tool integration, unit testing, CI/CD
understanding concepts of AI/LLMs and understanding the concepts of the technology/platformthe AI creates suggestions/solutions for
verifikation und verständnis des generierten Codes
verifying sources, actually understanding and not blindl copying
vertiefte Programmierkenntnisse
wissenschaftlich-kritische Reflexionsfähigkeit, Nachhaltigkeitsbewusstsein, technisches Verständnis, Frustrationstoleranz, Selbstwertgefühl, Ethikbewusstsein, Projektmanagementfähigkeiten
you need to be a good sw engineer/architect and can make decission what is good or bad, to avoid the AI produces technical debt instead of realy helping.