← Back to The Index Final Review Submission #16
Final Review

Review Nº 16

Calibrating Worker Trust in AI: A Multi-Dimensional Scoping Review on Professional Delegation
AuthorsSaverio Medici, Artemisia Lollo, Denis Alessandro Ciobanu, Matteo Vaccaneo, Bruno Germanis, Eugenio Costella, Matteo Fissore, Alessio Nicotra, Lorenzo Chiaramello and Matteo Rizzolo
30/30
Score
The revision delivers clear, substantive improvements — synthesis table, inline citations, expanded source pool, broader use-case coverage, and an explicit Gaps section addressing geography and longitudinal effects — but several consensus points remain open, notably operational thresholds for trust calibration, layer interactions, factor weighting, and a still-informal introduction.
Perfect Score

The Pros

7 Items
+
Synthesis table (Table 1) now maps factors to pillars and sources, directly answering R1 and R4.
+
Inline numeric citations are systematically used throughout, resolving R2's biggest concern.
+
Source base expanded from 18 to 47, with a PRISMA-inspired screening figure improving methodological transparency.
+
Use case reorganised around all five layers (Governance, Technical, Transparency, Psychological, Agency), responding to R1 and R9; it is also explicitly framed as a "projection", not empirical evidence.
+
Section 5 (Gaps) directly tackles geographic limitations (R3, R8) and de-skilling/longitudinal evidence (R3, R8).
+
AI tool-use disclosure (Table 2) is detailed and verifiable, addressing R3 and R7.
+
The "Responsibility Constraint" mentioned by R7 is preserved and the conclusion ties IDA, XAI, and HITL into a coherent thesis.

The Cons

8 Items
Trust calibration thresholds are promised in the abstract ("we identify the thresholds") but never operationalised; over-trust/under-trust per layer is still undefined (R3, R7).
Layer interactions and internal tensions between the five dimensions are not discussed (R6, R9).
Factor frequency / relative weight across the 47 sources is still missing — Table 1 lists them but does not rank them (R1, R4).
Governance layer remains thin: legal liability is acknowledged but not resolved; EU AI Act alignment, audit protocols, and rejection criteria are absent (R2, R3, R6, R8).
Use case is still a single bankruptcy-accountant scenario; no second profession or comparative analysis demonstrates generality (R4, R8).
Introduction retains informal phrasing ("incredible and powerful tools", "etcetera"), which R6 explicitly flagged.
Framework's own limitations are not stated; only literature limitations are (R8, R9).
SHAP/LIME are still referenced without a brief explanation for non-technical readers (R2).

Suggested Changes

12 Pointers
01
High
Location
Abstract
Issue
Claims "we identify the thresholds required for calibrated trust" but no thresholds are defined in the body
Suggested Fix
Either define concrete calibration thresholds (e.g., per-layer over-trust/under-trust signals) or rephrase the abstract to "we identify the dimensions along which trust must be calibrated"
02
High
Location
Section 1 (Introduction), opening paragraph
Issue
Tone is informal and rambling ("incredible and powerful tools", "etcetera in a much more expanded way"), reproducing the issue R6 flagged
Suggested Fix
Rewrite the first paragraph in a concise, formal academic register; lead with the research gap and the question, removing rhetorical flourishes
03
High
Location
Section 3 (Framework), end of layer descriptions
Issue
Layers are presented in isolation; no analysis of how they interact or where they conflict
Suggested Fix
Add a short subsection or paragraph titled "Inter-layer dynamics" discussing trade-offs (e.g., transparency vs. cognitive workload, governance vs. autonomy, psychological filter vs. technical robustness)
04
High
Location
Table 1
Issue
Lists factors and source citations but provides no weighting or frequency signal
Suggested Fix
Add a column with the count of supporting sources per factor, or bold the factors mentioned in ≥5 sources, so readers see which factors are most strongly supported
05
High
Location
Section 3, Layer 5 (Systemic Shield)
Issue
Legal liability and concrete governance practices remain thin
Suggested Fix
Add a paragraph on EU AI Act alignment (Art. 14 human oversight, Art. 9 risk management) and list at least three concrete governance practices: third-party algorithmic audits, documented rejection criteria, and HITL escalation protocols
06
High
Location
Section 4 (Worked Use Case)
Issue
Generality of the framework is asserted but only one profession is demonstrated, leaving R4 and R8 unresolved
Suggested Fix
Add a brief secondary mini-case (e.g., a radiologist or HR recruiter) — even one paragraph per layer — to show transferability across high- and medium-stakes settings
07
Medium
Location
Section 3, Layer 2
Issue
SHAP and LIME are introduced without explanation, blocking non-technical readers
Suggested Fix
Add one sentence each: SHAP attributes a prediction to its input features via Shapley values; LIME approximates a black-box model locally with an interpretable surrogate
08
Medium
Location
Section 5 (Gaps and Future Work)
Issue
Discusses limitations of the literature but not of the proposed MDTF itself
Suggested Fix
Add a paragraph titled "Limitations of the MDTF" stating that the framework is conceptual, has not been empirically validated as a whole, and lacks defined rejection criteria for AI outputs
09
Medium
Location
Section 4, "Application of the Calibration Framework"
Issue
The five layers are walked through but never tied back to a concrete calibration outcome (trust/no trust/conditional)
Suggested Fix
Close the use case with a one-paragraph "Calibration verdict" stating, layer by layer, whether the conditions are met and what residual risk remains
10
Medium
Location
Section 2.2 (Search Strategy)
Issue
R4 and R7 flagged a mismatch between described methodology and search log; the paper points to the appendix but does not summarise key search strings
Suggested Fix
Inline the actual Boolean search strings used (at least the most representative two or three) so methodology and appendix are visibly consistent
11
Low
Location
Section 6 (Conclusion)
Issue
Repeats the framework but does not return to the research question stated in 2.1
Suggested Fix
Open the conclusion by restating the Type-3 research question and explicitly answering it in one sentence before summarising the layers
12
Low
Location
References & in-text usage
Issue
Some long inline citation chains in the appendix (page 9) are presented without context, weakening the synthesis R7 asked for
Suggested Fix
Replace the bare citation list at the end with a one-line consensus note per pillar (e.g., "12 of 47 sources converge on accuracy as a primary antecedent")
Back to The Index
You aced it · 30/30
Read · React · Revise
Pros / Cons / Pointers
Final Review · Submission #16 The Index Grandi Sfide · 2026