Assistance Request for ConDA-gen-text-detection Code Implementation

[Doran-fans]

Background
I am a college student currently working on implementing the ConDA-gen-text-detection code from Amrita Bhattacharjee's GitHub repository. During this process, I have encountered some issues and would appreciate some guidance and assistance.

Process
I downloaded the data from TuringBench, specifically the TuringBench.zip file, and successfully processed all CSV files into real_dataset.jsonl and fake_dataset.jsonl files.
After completing the preprocessing steps, I generated the relevant scrambled text and proceeded to run the multi_domain_runner.py script, resulting in the creation of a .pt model file.
Problem
However, when attempting to evaluate the model using the evaluation.py script, I did not obtain the F1 score mentioned in Amrita Bhattacharjee's paper. I am using the robert-base coding tool for this implementation.

Request
I would greatly appreciate some insights into the correct approach for obtaining the results mentioned in the paper. There may be specific parameters or steps that I overlooked during the process.

Relevant Information
Encoder tool used: robert-base
Data source: TuringBench.zip file
Processed files: real_dataset.jsonl and fake_dataset.jsonl
Contact Information
Name: Ruifan Zhao
University: Mongolian University
Email: zhaomr314@gmail.com
Thank you for your time and assistance. Looking forward to your response.

[AmritaBh]

Hi Ruifan,

The ConDA model is supposed to be trained with data from one source generator and one target generator, not all the data from TuringBench at once (as described in the paper). After pre-processing the files from the two generators, you need to perform some text transformation (such as synonym replacement, as described in the paper). An example notebook for this is also provided in the repo. The files that you get after this step will be used for training. Please make sure to edit/change paths and directories and other parameters as necessary when you run the scripts.

[Doran-fans]

Dear Amrita Bhattacharjee, Since I have not modified your code, I suspect that there is a problem with the processing of the data set. I will compile my processing process into a pdf and provide it to you. I hope to get your help. If possible, please provide me with the compressed package of your data set. Thank you very much! Amrita Bhattacharjee ***@***.***> 于2023年12月18日周一 06:54写道：

Hi Ruifan, The ConDA model is supposed to be trained with data from one source generator and one target generator, not all the data from TuringBench at once (as described in the paper). After pre-processing the files from the two generators, you need to perform some text transformation (such as synonym replacement, as described in the paper). An example notebook for this is also provided in the repo. The files that you get after this step will be used for training. Please make sure to edit/change paths and directories and other parameters as necessary when you run the scripts. — Reply to this email directly, view it on GitHub <#1 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/A6UKW3XR6GO7XV3XK36H7U3YJ5Z3JAVCNFSM6AAAAABAXQERQSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQNJZGMYTENRWGE> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[image: Mailtrack] <https://mailtrack.io?utm_source=gmail&utm_medium=signature&utm_campaign=signaturevirality11&> Sender notified by Mailtrack <https://mailtrack.io?utm_source=gmail&utm_medium=signature&utm_campaign=signaturevirality11&> 24/01/18 22:43:22