±Forensic Focus Partners

Become an advertising partner

±Your Account


Username
Password

Forgotten password/username?

Site Members:

New Today: 0 Overall: 36232
New Yesterday: 4 Visitors: 116

±Follow Forensic Focus

Forensic Focus Facebook PageForensic Focus on TwitterForensic Focus LinkedIn GroupForensic Focus YouTube Channel

RSS feeds: News Forums Articles

±Latest Articles

±Latest Videos

±Latest Jobs

ACPO Principles Revised

Computer forensics discussion. Please ensure that your post is not better suited to one of the forums below (if it is, please post it there instead!)
Reply to topicReply to topic Printer Friendly Page
Forum FAQSearchView unanswered posts
Page 1, 2, 3, 4, 5, 6, 7, 8, 9, 10  Next 
  

tootypeg
Senior Member
 

ACPO Principles Revised

Post Posted: Nov 05, 19 20:38

I thought I would post this as its something ive been working on and the exact issue has popped up in another recent thread by GumStick. I have long thought that despite ACPO not existing anymore we constantly will refer back to the 4 governing principles of digital evidence which I dont think suffer from being outdated. - This has also been echo'd by the posters in that thread.

I have been putting together a piece of work on this for a while now, but thought I would share what I think are my proposed set of new 8 principles for digital investigation which I think could now be more appropriate than the traditional 4.

I would be super interested in thoughts on this and hopefully it is also helpful in the other thread.


Principle 1: “Any course of investigatory action undertaken by the practitioner must first be agreed upon by an appropriate authority, who themselves must have full knowledge and insight into any agreed course of action in order to adopt responsibility for subsequent investigatory decision making.”

Principle 2:- “A practitioner must understand those laws, policies and principles applicable to their given inquiry, which define the scope of their investigatory powers. Practitioners must evidence adherence to these, and operate within their confines at all times”.

Principle 3:- “A practitioner should make all reasonable efforts to identify sources of potential evidence relevant to their investigation, taking into account the concepts of proportionality and necessity in regards to any device seized/interrogated. All justifiable measures must be taken to limit both collateral intrusion and any disruption caused by their investigation.”

Principle 4:- “A practitioner should only access data on a digital device using a suitable method. Suitability is determined by the following:
A known and accepted method, which has been subject to peer and field-wide review.
A developed novel method providing suitable testing and validation has been undertaken in order to verify its functionality.
In either case, the practitioner must understand the any methods used and be able to explain their function.”

Principle 5:- “A practitioner should take all reasonable steps to preserve the integrity of any device(s) subject to investigation during the course of their examination.”

Principle 6:- “Methods of access which compromise the initial state of digital data on a device must be utilised as a last resort. Where such methods are implemented, the implications of their use must be both understood and capable of explanation by the practitioner.”

Principle 7:- “All extracted and interpreted data deemed to be ‘digital evidence’ must have undergone robust testing and validation using accepted testing methods and peer review in order to verify accuracy.”

Principle 8:- “All stages of a practitioners investigation must be documented, forming an audit trail which can be used to describe those processes implemented by the practitioner to a third party, and where necessary and possible, allowing these procedures to be repeated in order to obtain comparable results.”
 
 
  

thefuf
Senior Member
 

Re: ACPO Principles Revised

Post Posted: Nov 05, 19 23:44

- tootypeg
Any course of investigatory action undertaken by the practitioner must first be agreed upon by an appropriate authority


If there is one.

A practitioner should only access data on a digital device using a suitable method


Not only on a digital device, but also in a system consisting of two or more digital devices (acting as a whole).

A known and accepted method, which has been subject to peer and field-wide review.
A developed novel method providing suitable testing and validation has been undertaken in order to verify its functionality.
In either case, the practitioner must understand the any methods used and be able to explain their function.


There are acquisition methods that:
- are not properly documented, [and/or]
- are often misunderstood (or at least not well-understood) by practitioners who utilize these methods.

Some related threads as examples:
www.forensicfocus.com/...c/t=15155/
www.forensicfocus.com/...c/t=15061/

TL;DR:
Why do we need to insert a microSD card into a Samsung device during the bootloader acquisition with UFED? No, it's not used as a buffer. No, it's not used as a destination storage device. No, the phone doesn't boot from that card. No, Cellebrite doesn't provide an accurate explanation. But if you have some background in binary exploitation and you are not afraid of EULA violations, you can try to reverse engineer the process and get the answer.

So:
A developed novel method providing suitable testing and/or validation has been undertaken in order to verify its functionality.

to preserve the integrity of any device(s) subject to investigation


And don't forget about the integrity of digital data stored on these devices.

Methods of access which compromise the initial state of digital data on a device must be utilised as a last resort. Where such methods are implemented, the implications of their use must be both understood and capable of explanation by the practitioner.


And reasonable actions should be taken to preserve and/or document digital data which may become inaccessible in the future, even when this data is not directly affected by methods in question.

Examples (mobile device forensics when there is no way to acquire a full image): expired entries in web browsing history, applications that refuse to run because an update is required after a specific date.

All extracted and interpreted data deemed to be ‘digital evidence’ must have undergone robust testing and validation using accepted testing methods and peer review in order to verify accuracy.


This is impossible. A simple question: is it possible to properly validate a hard disk drive acquisition process/tool? In most cases, my answer is "no". Because labs usually don't engage in reverse engineering.  
 
  

Rich2005
Senior Member
 

Re: ACPO Principles Revised

Post Posted: Nov 06, 19 08:08

I'm going to be critical here but please take it as constructive criticism as I'm not trying to shoot you down.

I think the beauty of the 4 ACPO principles was their brevity. In number, length, and lack of prescriptiveness (may have just invented a word there).

There's danger you're veering off into the same problem as ISO17025 with trying to pigeon hole everything unnecessarily.

I think the problem with point one is it's too definitive and prescriptive. The officer in charge or authority may often not "have full knowledge and insight" and be very non-technical. I'm also not sure "any" course of action should necessarily be agreed as, to the letter of that, you'd be forever going back to the OIC (or similar) for the tiniest thing you're doing.

The problem with point two is it's further setting up DF practitioners for a fall if something has changed without their knowledge. It also technically doesn't limit the implication to DF related laws and is saying they should understand all laws in their case. That's not necessary and someone wouldn't need to be an expert in tax law to produce reliable evidence for a financial investigation.

Principle three (the start) might not apply to many situations and be outside of the remit of an examiner. They might be dealing with one device as part of a larger investigation or be part of a larger team and the identification phase/responsibility falls to someone else more senior.

You'll be unsurprised to know I don't like principle 4 or principle 7. It's very ISO17025. I have big problems with the "lots of us use it so it's fine" logic or the "I've performed some limited testing so it's fine" logic. I think ISO17025 is a dangerous thing in DF for this reason, and it's the illusion of reliability, where it demonstrably doesn't exist, and the skill of the examiner will be far more important in how they identify and report their findings, check their findings where possible, spot problems or potential problems, give caveats where appropriate, etc.

Principle 8, being basically the same as ACPO principle 3, my thoughts are just the same as posted on the other thread (which sparked your post).  
 
  

jaclaz
Senior Member
 

Re: ACPO Principles Revised

Post Posted: Nov 06, 19 08:48

- tootypeg
I thought I would post this as its something ive been working on and the exact issue has popped up in another recent thread by GumStick. I have long thought that despite ACPO not existing anymore we constantly will refer back to the 4 governing principles of digital evidence which I dont think suffer from being outdated. - This has also been echo'd by the posters in that thread.


Only for the record, the thread is this one, started by member GumStickStorage:
www.forensicfocus.com/...c/t=18147/

And the posters that commented on the ACPO principles actually - including yourself (and for the very little that counts myself) has been actually critical of them (hence the *need* to have them revisited).

More specifically the original 4 ACPO principles are (IMHO) still very good in theory but fail in the practical application.

Your new set of 8 principles (while being as well good in theory) seem to me like falling in exactly the same issues in practice, let's call them NG Confused (Next Generation).



Principle NG#1 -> Nice, please make a list of appropriate authorities you personally know that actually understand anything about digital forensics methods

Principle NG#2 -> Not very different from ACPO principle #2

Principle NG#3 -> Define "reasonable" and "justifiable", an while you are at it, define also "proportionality" and "necessity", though I am not really-really qualified for these comments Shocked , these seems to me like a good way to invite defense attorneys to a party.

Principle NG#4 -> Nice. please make a list of the major or most common forensics softwares currently in use that have detailed the methods they use and where these methods have been peer reviewed.

Principle NG#5 -> Sure, that is ACPO #1, but - again - there is the definition of "reasonable" missing.

Principle NG#6 -> This is the corollary to NG#5 above, but as said on the other thread, the practitioner should have understood and be capable of explaining the methods used anyway

Principle NG#7 -> It seems like essentially a repetition of (or a corollary to) NG#4 above?

Principle NG #8 -> Fine, this is ACPO #3, with a twist, who/why/when is determined that repeating is "necessary"?

In this set the concept of ACPO principle #4 (which is IMHO important) seems like missing?

jaclaz
_________________
- In theory there is no difference between theory and practice, but in practice there is. - 
 
  

tootypeg
Senior Member
 

Re: ACPO Principles Revised

Post Posted: Nov 06, 19 09:10

Super interesting replies to this, all comments well man, I think its good to be able to pick holes in this and actually see any/all issues so everything is welcome!

Thefuf:-

I see your point about an appropriate authority, I guess this is a catch all phrase that covers that person(s) who sanction an investigation. Surely in most cases, there will be such a person/body?

In regards to accessing digital data on a device - I guess phraseology should encompass non-local stored data - which I would still argue is on a device and therefore the terminology technically would stand?...maybe clutching at straws.

The acquisition method point is valid and overlaps with Rich2005. I guess we should understand everything that we do, but often accept that as its done by others its ok. Arguably its an unacceptable stance, but how do we solve this solution. We cant weaken a principle to suit the field because we currently dont do something? So surely the principle has to be that we must understand the process - where in reality how we achieve this is the problem to be addressed. But I see your point, good example.

This is impossible. A simple question: is it possible to properly validate a hard disk drive acquisition process/tool? In most cases, my answer is "no". Because labs usually don't engage in reverse engineering.


Possibly, but should this mean that we shouldn't have it as a principle? I mean if we could do it effectively and efficiently we surely would do this validation - therefore I could argue that it should be a principle and the burden of achieving it be something we have to address?

[/b]I just want to say that hearing what I am typing I am not trying to cause us more issues, just trying to play devils advocate (correct phrase?!) [b]

Rich2005:-

Do you think the original 4 principles now are too generic/lacking in content that they no longer offer anything more than an anecdote? I dont think they arnt applicable, but Im thinking that things have shifted with privacy, quality and procedural issues now more strongly in play?

I think the problem with point one is it's too definitive and prescriptive. The officer in charge or authority may often not "have full knowledge and insight" and be very non-technical. I'm also not sure "any" course of action should necessarily be agreed as, to the letter of that, you'd be forever going back to the OIC (or similar) for the tiniest thing you're doing.


I hear your point here, maybe it needs moderating as a principle. I guess the essence of this is that before we make a decision, we should have permission and the permission granter should understand what they are granting permission for? In reality, it could be tough to implement, but is it also not sensible in some respects?

The problem with point two is it's further setting up DF practitioners for a fall if something has changed without their knowledge. It also technically doesn't limit the implication to DF related laws and is saying they should understand all laws in their case. That's not necessary and someone wouldn't need to be an expert in tax law to produce reliable evidence for a financial investigation.


Good point, I guess again language moderation. Maybe understanding of the investigatory laws that govern their actions rather than those of the laws of the suspect offence under investigation?


You'll be unsurprised to know I don't like principle 4 or principle 7. It's very ISO17025. I have big problems with the "lots of us use it so it's fine" logic or the "I've performed some limited testing so it's fine" logic. I think ISO17025 is a dangerous thing in DF for this reason, and it's the illusion of reliability, where it demonstrably doesn't exist, and the skill of the examiner will be far more important in how they identify and report their findings, check their findings where possible, spot problems or potential problems, give caveats where appropriate, etc.


Laughing I do agree. I think reliance on external testing and validation is risky. I dont like it. But I was curious to see what the reception of this stance may be. A principle that suggests the practitioner should self test/validate may be more appropriate and I prefer. In reality, as noted to Thefuf, it might not be practical - but should this stop it from being a principle?  
 
  

Rich2005
Senior Member
 

Re: ACPO Principles Revised

Post Posted: Nov 06, 19 09:42

- tootypeg
Do you think the original 4 principles now are too generic/lacking in content that they no longer offer anything more than an anecdote? I dont think they arnt applicable, but Im thinking that things have shifted with privacy, quality and procedural issues now more strongly in play?


I think they're suitably generic (albeit you could advocate minor tweaks) and there's a far bigger issue that needs addressing in terms of forensic regulation and ISO which is a mess being forced upon us.

- tootypeg
I hear your point here, maybe it needs moderating as a principle. I guess the essence of this is that before we make a decision, we should have permission and the permission granter should understand what they are granting permission for? In reality, it could be tough to implement, but is it also not sensible in some respects?


What you're trying to get at isn't wrong. However it's simply the real-world practicalities that need to be considered. I think the ACPO principle point about the OIC being in charge is essentially getting at the same thing.

- tootypeg
Good point, I guess again language moderation. Maybe understanding of the investigatory laws that govern their actions rather than those of the laws of the suspect offence under investigation?


Yes, although I'd argue this is a slightly dangerous thing to be including, as I'd (on a separate topic) like to hear people's views on the collection of cloud data when on a warrant, for example. I've seen it done and I also know many people won't because they don't believe it's covered under the scope of the warrant (ie stored elsewhere). I think this is one of the many areas of DF that could be improved by having a central body tasked with improving the state of the field (whether testing tools, providing guidance on topics such as this, providing guidance on mobile phone seizure perhaps, etc). I think the biggest problem in DF is problems and rules being created for examiners but not enough focus on actually improving the quality of evidence and its collection (because ISO17025 most certainly doesn't do that - if anything the opposite).


- tootypeg
Laughing I do agree. I think reliance on external testing and validation is risky. I dont like it. But I was curious to see what the reception of this stance may be. A principle that suggests the practitioner should self test/validate may be more appropriate and I prefer. In reality, as noted to Thefuf, it might not be practical - but should this stop it from being a principle?


My preference (as above and in other persons) is a central body that works on creating guidance, on laws, procedures, and helps with tool vetting. Rather than extra principles, or rules, that shift the focus (blame) on to the examiners, but don't solve the major problems or areas that could be improved within DF.

DF is such a massive field now, and probably more crucial than any other type of evidence, that it warrants serious funding. The single biggest risk in the field is, in my view, not rogue examiners or incompetence. It's lack of funding as a result the lack of time spent on cases whether inside law enforcement or not (on top of the lack of guidance, central tool testing, etc). You can add lack of training for all types of law enforcement officers (and barristers) into this too. There have been tiny steps to address this in both cases but it's generally been p*ssing in the wind to put it bluntly. However this has been the case for a long time and it's a governmental level issue that frankly they're just not aware enough of (and even if they were they probably wouldn't treat seriously enough to address).  
 
  

jaclaz
Senior Member
 

Re: ACPO Principles Revised

Post Posted: Nov 06, 19 10:46

- Rich2005

My preference (as above and in other persons) is a central body that works on creating guidance, on laws, procedures, and helps with tool vetting.


Which would in theory create the need of a sort of NIST (and its CFTT):
www.nist.gov/itl/ssd/s...ogram-cftt
and its:
www.dhs.gov/science-an...tt-reports
and:
toolcatalog.nist.gov/

which - understandably - even when a test/report exists (and it is exhaustive), often is related to a previous version of the software.

Moreover, again understandably, there are quite a few tests for "disk imaging" (which is - or should be - something that *any* practitioner can validate himself/herself):
www.dhs.gov/publicatio...sk-imaging
and only a few (actually 2 FTK and Encase) for "Windows Registry Forensic Tool":
www.dhs.gov/publicatio...ensic-tool

Assuming that these latter are the "main" or "big enough" players in the field (and all in all Windows Registry parsing should be IMHO among the "easy" ones among the many needs of an investigations and even if the tests are about fairly recent versions) they are not "perfect":
FTK:
  • The tool incorrectly reported a QWORD value.
  • The tool did not process hive files generated by hivex library.
  • The tool did not report several big-data values in a v1.5 hive file.

Encase:
  • The tool was terminated without any notification when it processed a tree structure with a large number of levels (about 1 million) in an experimental hive file.
  • Long value names (16,383 bytes and more) were not reported.
  • The tool did not report UTF-16LE characters properly.
  • The tool did not identify unusual ASCII characters (between 0x04 and 0x0D) of key and value names.
  • The ‘Tree’ and ‘Table’ panes of the tool operated differently when showing ASCII and UTF-16LE characters

So, again, it is likely that in practice a similar approach is simply not doable or pretty much unuseful/incomplete.

- Rich2005

Rather than extra principles, or rules, that shift the focus (blame) on to the examiners, but don't solve the major problems or areas that could be improved within DF.

Yep Smile , and both the original ACPO principles and the NG revised version, if you look at them from some distance can be condensed in a single meta-principle Shocked :
Do the right thing.

Wink



jaclaz
_________________
- In theory there is no difference between theory and practice, but in practice there is. - 
 

Page 1 of 10
Page 1, 2, 3, 4, 5, 6, 7, 8, 9, 10  Next