Update SMSmissingROWIDs.py

[MetadataForensics]

Updated for both new artifact format and query changes. Additional information is available here: https://github.com/MetadataForensics/RowIDetective

[JamesHabben]

Great information on your repo page, and thanks for referencing the LEAPPs too!

A couple notes:

Timestamp adjustment

1. Would you consider adding a raw timestamp output column for any existing date columns to this query? We are trying to get artifacts to use our api functions for timestamp conversions for consistency in timestamp handling and output. I recognize that it is much easier to maintain these queries if you can copy/paste in all the different places it is being used, so I don't want to ask that you make changes to the query just for iLEAPP. I thought an additional raw column for any dates could be easy enough to maintain, and it would even provide a reference point for examiners to validate the data by doing the conversion calculation manually.
2. We are also working toward being able to process and show sub-second resolution on timestamps, and I note that you have a condition in the query to detect that condition and simplify. If you add a raw timestamp output column and use our conversion api functions, we can take that in and provide the sub-second data immediately when the core code is ready for it.
   see: lines 60-64 as a reference.

Test Data

Do you know of any publicly available valid test data that we can use in our test library to allow for continuous testing?

[SQLMcGee]

I submitted this pull request, this is just my other account.

Glad the repo is helpful, this has been a fun little project.

I don't think those changes will be a problem at all, I started into it but caught up with some other things. It's interesting the changes and improvements, it's been a minute since I submitted anything here - hence, all the other changes to this artifact as well.

I don't know a test dataset offhand that includes this newest change.. basically this covers the most recent sent/received iMessages being removed making a difference between the max ROWID and the sqlite_sequence number for the message table. You can recreate this scenario by adjusting the seq so it's higher and zipping the appropriate folder structure. It's a quick test in a bind.

But, again, I'll keep looking at the timestamps.

Thanks for all you do!

[MetadataForensics]

I reviewed timestamp conversions under iLEAPP/scripts/ilapfuncs.py - is there currently any other handling of timestamp values that may be 9 digit (seconds) or 18 digit (nanoseconds)? I may have missed something that you have defined. Having not seen anything in ilapfuncs.py that accounts for the possible timestamp difference and going away from the case statement I originally had the def fix_ts(val) worked here, for just this artifact. I've been working this query for some time now and Apple has adjusted timestamp recordings within the sms.db over the years so handling both possible timestamps is more universal for datasets.
Also, added in columns of just the raw message.date timestamp values should the users want to validate the data without going to the database itself.

[JamesHabben]

@Johann-PLW has done the work on those timestamp conversions and would know better