Feature/user models

README.md

@@ -1,41 +1,69 @@
-# LangPro Annotator
-
-[![Actions Status](https://github.com/CentreForDigitalHumanities/langpro-annotator/workflows/Unit%20tests/badge.svg)](https://github.com/CentreForDigitalHumanities/langpro-annotator/actions)
-
-An annotation tool for LangPro, a tableau-based theorem prover for natural logic and language.
-
-*This repository is a work in progress, and the README is not yet complete!*
-
-## Introduction
-
-*In this section, provide an overview of your code and describe the project in which the code was developed. Highlight the purpose, scope, and potential uses of your code. Also, consider including links to relevant publications or resources that provide additional context.*
-
-## Getting started
-
-LangPro Annotator is a web application: it can be accessed using a web browser.
-
-*If you are hosting LangPro Annotator anywhere, provide a URL and any additional info.*
-
-You can also run LangPro Annotator locally or host it yourself. Be aware that this is a more advanced option. See [CONTRIBUTING.md](./CONTRIBUTING.md) for information about setting up a local server or configuring deployment.
-
-## Usage
-
-*Provide information about what LangPro Annotator can be used for. If you have written a user manual, this is the place to link it!*
-
-## Development
-
-To get started with developing LangPro Annotator, see [CONTRIBUTING.md](./CONTRIBUTING.md).
-
-## Licence
-
-This work is shared under a BSD 3-Clause licence. See [LICENSE](./LICENSE) for more information.
-
-## Citation
-
-To cite this repository, please use the metadata provided in [CITATION.cff](./CITATION.cff).
-
-## Contact
-
-LangPro Annotator is developed by [Research Software Lab, Centre for Digital Humanities, Utrecht University](https://cdh.uu.nl/about/research-software-lab/).
-
-*Include contact information. You can also provide clear instructions for how users can provide feedback, contribute, or suggest improvements to your work.*
+# LangPro Annotator
+
+[![Actions Status](https://github.com/CentreForDigitalHumanities/langpro-annotator/workflows/Unit%20tests/badge.svg)](https://github.com/CentreForDigitalHumanities/langpro-annotator/actions)
+
+An annotation tool for LangPro, a tableau-based theorem prover for natural logic and language.
+
+*This repository is a work in progress, and the README is not yet complete!*
+
+## Introduction
+
+*In this section, provide an overview of your code and describe the project in which the code was developed. Highlight the purpose, scope, and potential uses of your code. Also, consider including links to relevant publications or resources that provide additional context.*
+
+## Getting started
+
+LangPro Annotator is a web application: it can be accessed using a web browser.
+
+*If you are hosting LangPro Annotator anywhere, provide a URL and any additional info.*
+
+You can also run LangPro Annotator locally or host it yourself. Be aware that this is a more advanced option. See [CONTRIBUTING.md](./CONTRIBUTING.md) for information about setting up a local server or configuring deployment.
+
+## Usage
+
+*Provide information about what LangPro Annotator can be used for. If you have written a user manual, this is the place to link it!*
+
+## Development
+
+To get started with developing LangPro Annotator, see [CONTRIBUTING.md](./CONTRIBUTING.md).
+
+## Permission overview
+
+Apart from superusers/admins, the application has three user roles with different permissions. Users with the "Master Annotator" role have full permissions, including managing users and problem visibility/status. Users with the "Annotator" role can browse and annotate both gold and silver problems. Users without any assigned role are considered "Visitors" and can only browse gold problems.
+
+Users can become Annotators or Master Annotators by adding them to the respective user groups in the Django admin interface.
+
+The matrix below shows the permissions for each role. Not all permissions are currently implemented.
+
+(Last updated: November 14th, 2025)
+
+|                            | Visitor | Annotator | Master Annotator |
+| -------------------------- | ------- | --------- | ---------------- |
+| Browse gold problems       | Yes     | Yes       | Yes              |
+| Browse silver problems     | No      | Yes       | Yes              |
+| Edit KB items              | No      | Yes       | Yes              |
+| Add labels                 | No      | Yes       | Yes              |
+| Remove own labels          | No      | Yes       | Yes              |
+| Remove other users' labels | No      | No        | Yes              |
+| Add problems               | No      | No        | Yes              |
+| Copy problems              | No      | No        | Yes              |
+| Update user problems       | No      | No        | Yes              |
+| Delete problems            | No      | No        | Yes              |
+| Edit existing problems     | No      | No        | Yes              |
+| See hidden problems        | No      | No        | Yes              |
+| Silver/gold problems       | No      | No        | Yes              |
+| Hide/unhide problems       | No      | No        | Yes              |
+| Manage users               | No      | No        | Yes              |
+
+## Licence
+
+This work is shared under a BSD 3-Clause licence. See [LICENSE](./LICENSE) for more information.
+
+## Citation
+
+To cite this repository, please use the metadata provided in [CITATION.cff](./CITATION.cff).
+
+## Contact
+
+LangPro Annotator is developed by [Research Software Lab, Centre for Digital Humanities, Utrecht University](https://cdh.uu.nl/about/research-software-lab/).
+
+*Include contact information. You can also provide clear instructions for how users can provide feedback, contribute, or suggest improvements to your work.*

backend/problem/management/commands/import_fracas.py

@@ -43,6 +43,9 @@ def _annotate_section_subsections(tree: ET.ElementTree) -> None:
 
         root = tree.getroot()
 
+        if root is None:
+            raise ValueError("The XML file is empty or malformed.")
+
         for element in root:
             if element.tag == "comment" and element.attrib.get("class") == "section":
                 current_section = element.text.strip() if element.text else None
@@ -107,15 +110,15 @@ def import_fracas_problems(self, fracas_path: str) -> None:
                 )
                 skipped += 1
                 continue
-            hypothesis = Sentence.objects.create(
+            hypothesis = Sentence.objects.update_or_create(
                 text=FracasData._text_from_element(hypothesis_node)
-            )
+            )[0]
 
             premises = []
             premise_nodes = problem.findall("p")
             for node in premise_nodes:
                 if node.text:
-                    premises.append(Sentence.objects.create(text=node.text.strip()))
+                    premises.append(Sentence.objects.update_or_create(text=node.text.strip())[0])
 
 
             entailment_label = self.ENTAILMENT_LABELS.get(

backend/problem/management/commands/import_sick.py

@@ -57,9 +57,8 @@ def import_sick_problems(self, sick_path: str) -> None:
 
                 extra_data = SickData.import_data(problem)
 
-                premise = Sentence.objects.create(text=problem["sentence_A"])
-
-                hypothesis = Sentence.objects.create(text=problem["sentence_B"])
+                premise = Sentence.objects.update_or_create(text=problem["sentence_A"])[0]
+                hypothesis = Sentence.objects.update_or_create(text=problem["sentence_B"])[0]
 
                 problem = Problem.objects.create(
                     dataset=Problem.Dataset.SICK,

backend/problem/management/commands/import_snli.py

@@ -1,4 +1,5 @@
 import csv
+from typing import Literal
 
 from django.core.management.base import BaseCommand
 from tqdm import tqdm
@@ -53,7 +54,9 @@ def handle(self, *args, **options):
 
         self.import_snli_problems(snli_paths)
 
-    def import_snli_problems(self, snli_paths: list[tuple[str, str]]) -> None:
+    def import_snli_problems(
+        self, snli_paths: list[tuple[Literal["dev", "train", "test"], str]]
+    ) -> None:
         """
         Import SNLI 1.0 problems from a list of SNLI TSV files and enter them into the database.
         """
@@ -99,9 +102,13 @@ def import_snli_problems(self, snli_paths: list[tuple[str, str]]) -> None:
 
                         extra_data = SNLIData.import_data(problem, subset)
 
-                        premise = Sentence.objects.create(text=problem["sentence1"])
+                        premise = Sentence.objects.update_or_create(
+                            text=problem["sentence1"]
+                        )[0]
 
-                        hypothesis = Sentence.objects.create(text=problem["sentence2"])
+                        hypothesis = Sentence.objects.update_or_create(
+                            text=problem["sentence2"]
+                        )[0]
 
                         new_problem = Problem.objects.create(
                             dataset=Problem.Dataset.SNLI,

backend/problem/migrations/0005_problem_base.py

@@ -0,0 +1,26 @@
+# Generated by Django 4.2.20 on 2025-09-22 11:47
+
+from django.db import migrations, models
+import django.db.models.deletion
+
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        ("problem", "0004_sentence_remove_problem_premises_knowledgebase_and_more"),
+    ]
+
+    operations = [
+        migrations.AddField(
+            model_name="problem",
+            name="base",
+            field=models.ForeignKey(
+                blank=True,
+                help_text="The base problem from which this problem was derived, if any.",
+                null=True,
+                on_delete=django.db.models.deletion.SET_NULL,
+                related_name="derived_problems",
+                to="problem.problem",
+            ),
+        ),
+    ]

backend/problem/migrations/0006_alter_problem_options.py

@@ -0,0 +1,26 @@
+# Generated by Django 4.2.27 on 2025-12-08 13:23
+
+from django.db import migrations
+
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        ("problem", "0005_problem_base"),
+    ]
+
+    operations = [
+        migrations.AlterModelOptions(
+            name="problem",
+            options={
+                "permissions": [
+                    ("view_gold_problems", "Can view problems marked as gold"),
+                    ("view_silver_problems", "Can view problems marked as silver"),
+                    ("view_hidden_problems", "Can view hidden problems that are marked as hidden"),
+                    ("copy_problems", "Can copy problems"),
+                    ("change_problem_status", "Can change status of a problem (gold/silver)"),
+                    ("change_problem_visibility", "Can change problem visibility (hidden/visible)"),
+                ]
+            },
+        ),
+    ]

backend/problem/models.py

@@ -28,6 +28,15 @@ class EntailmentLabel(models.TextChoices):
         default=Dataset.USER,
     )
 
+    base = models.ForeignKey(
+        "self",
+        on_delete=models.SET_NULL,
+        null=True,
+        blank=True,
+        related_name="derived_problems",
+        help_text="The base problem from which this problem was derived, if any.",
+    )
+
     premises = models.ManyToManyField(
         Sentence,
         related_name="premise_problems",
@@ -47,6 +56,16 @@ class EntailmentLabel(models.TextChoices):
 
     extra_data = models.JSONField()
 
+    class Meta:
+        permissions = [
+            ("view_gold_problems", "Can view gold problems"),
+            ("view_silver_problems", "Can view silver problems"),
+            ("view_hidden_problems", "Can view hidden problems"),
+            ("copy_problems", "Can copy problems"),
+            ("change_problem_status", "Can change problem status"),
+            ("change_problem_visibility", "Can change problem visibility"),
+        ]
+
     def get_index(self, qs: QuerySet) -> int | None:
         """
         Get the index of this Problem in a given queryset of problems, ordered by pk.
@@ -56,8 +75,6 @@ def get_index(self, qs: QuerySet) -> int | None:
         except Exception as e:
             logger.exception(f"Error getting index for problem {self.pk}: {e}")
             return None
-
-
 class KnowledgeBase(models.Model):
     class Relationship(models.TextChoices):
         EQUAL = "equal", "Equal"

backend/problem/serializers.py

@@ -62,6 +62,7 @@ class Meta:
             "entailmentLabel",
             "extraData",
             "kbItems",
+            "base",
         ]
 
     def get_premises(self, problem):
@@ -104,6 +105,7 @@ def create(self, validated_data: dict) -> Problem:
         )[0]
 
         problem = Problem.objects.create(
+            base_id=validated_data.get("base", None),
             hypothesis=hypothesis_sentence,
             dataset=Problem.Dataset.USER,
             # TODO: Determine entailment label based on LangPro parser output.
@@ -132,6 +134,19 @@ def update(self, instance: Problem, validated_data: dict) -> Problem:
         instance.hypothesis = Sentence.objects.get_or_create(
             text=validated_data["hypothesis"],
         )[0]
+
+        validated_base_id = validated_data.get("base", None)
+        if validated_base_id is None:
+            instance.base = None
+        else:
+            try:
+                base_problem = Problem.objects.get(id=validated_base_id)
+            except Problem.DoesNotExist:
+                raise serializers.ValidationError(
+                    f"Base problem with ID {validated_base_id} does not exist."
+                )
+            instance.base = base_problem # type: ignore
+
         instance.save()
 
         premise_sentences = [
@@ -188,6 +203,8 @@ class ProblemInputSerializer(serializers.Serializer):
         many=True, allow_empty=True, help_text="List of knowledge base items"
     )
 
+    base = serializers.IntegerField(required=False, allow_null=True)
+
     def validate_id(self, value):
         """Validate that the Problem ID, if provided, exists and belongs to a user-created problem."""
         if value is not None:
@@ -198,3 +215,12 @@ def validate_id(self, value):
                     f"Problem with ID {value} does not exist."
                 )
         return value
+
+    def validate_base(self, value):
+        """Validate that the base problem ID exists if provided."""
+        if value is not None:
+            if not Problem.objects.filter(id=value).exists():
+                raise serializers.ValidationError(
+                    f"Base problem with ID {value} does not exist."
+                )
+        return value

backend/problem/views/problem.py

@@ -12,12 +12,30 @@
 )
 from problem.models import Problem
 from problem.serializers import ProblemInputSerializer, ProblemSerializer
+from rest_framework.permissions import IsAuthenticated, IsAuthenticatedOrReadOnly
+
+
+class CreateProblemPermission(IsAuthenticated):
+    def has_permission(self, request, view):
+        return super().has_permission(request, view) and request.user.can_create_problem
+
+
+class EditProblemPermission(IsAuthenticated):
+    def has_permission(self, request, view):
+        return super().has_permission(request, view) and request.user.can_edit_problem
 
 
 class ProblemView(ModelViewSet):
     queryset = Problem.objects.all()
     serializer_class = ProblemSerializer
 
+    def get_permissions(self):
+        if self.action == "create":
+            return [CreateProblemPermission()]
+        if self.action == "partial_update":
+            return [EditProblemPermission()]
+        return [IsAuthenticatedOrReadOnly()]
+
     def list(self, request: Request) -> Response:
         """
         Lists all Problems in the database, with optional filtering.
@@ -55,7 +73,7 @@ def _get_problem_response(self, request: Request, pk: int | None) -> Response:
         qs = self.get_queryset()
 
         if filters is not None:
-            qs = qs.filter(filters)
+            qs = qs.filter(filters).distinct()
 
         problem = None
         if pk is not None:
@@ -111,7 +129,7 @@ def _handle_update_create_problem(
         problem_serializer = ProblemSerializer()
 
         if problem_id is None:
-            problem = problem_serializer.create(validated_input)
+            problem = problem_serializer.create(validated_input)  # type: ignore
             status = HTTP_201_CREATED
         else:
             problem_instance = get_object_or_404(
@@ -123,4 +141,3 @@ def _handle_update_create_problem(
             status = HTTP_200_OK
 
         return Response({"id": problem.pk}, status=status)
-