Some DFIR tools are terrible…if not used correctly.
I saw an engaging discussion online about tool choices that inspired this post about tools. I particularly enjoyed how someone gave an example of how tools are referred. I changed the example a little, but the more I look at it, the more I can remember this happening all the time:
Person 1: What tool can I use for “X”?
Person 2: Use this one.
Person 3: No, this one is better.
Person 4: But I like this other one better.
Person 5: That one sucks. Use this one instead.
Person 6: Why don’t you write your own tool?
Person 7: What’s wrong with my tool?
Person 1: Uh...thx?
This is great advice if:
-The question included specific details of the issue to be solved, and
-The tool(s) recommended can do the task, and
-The user knows how to use the tool.
This is bad advice if:
-The question was poorly framed, or
-The tools suggested don’t fit the need because the need was not described accurately, or
-The user doesn’t know how to effectively use the tool.
I also see complaints online about some DFIR tools, like:
-Why doesn’t this tool do encryption?
-How come email is so difficult to do with this tool?
-Why is the reporting so bad with this tool?
-How come this tool doesn’t find what I need it to find?
-Who validated this tool?
The solution to all of this is simple:
-Clearly define your forensic problem.
-Choose a tool designed to handle that problem.
-Use the tool correctly.
The things to not do:
-Don’t use a tool without knowing how to use that tool.
-Don’t use a tool without personally making sure that it works.
-Don’t use a tool that is not designed to do what you want it to do.
This all sounds so easy, but with the wide range of software available, it is easy to be overwhelmed with choices. Sometimes we fall in love with one tool and want it to do everything, even things that it may not be best suited to do or maybe not even designed to do at all. Sometimes we avoid a tool just because we don’t like the interface design. And many times we use tools without fully understanding what they are doing, what they are capable of doing, and just as important, what the tools are incapable of doing.
-Clearly define your forensic problem.
* Which OS, artifacts, etc…
* Desired output (depth of analysis, reporting, etc…).
-Choose a tool designed to handle that problem.
* Round peg in a round hole (don’t force a tool to do what it is not designed to do).
* Updated, maintained, used by the community, good reputation, etc…
-Use the tool correctly.
* Read the manual and/or take a class in that tool, and/or ask someone for guidance.
* Test it.
* Use it as designed for the problem designed to handle.
I have found that tools may have the same generic name and claim to do the same generic thing but are actually extremely far apart in what the tools actually do. Without knowing the scope and limitations of tools, you can miss everything in an analysis and not even know it. Or you can miss something so glaringly simple as to discredit your entire analysis, just because you didn’t employ an appropriate tool or maybe didn’t use an appropriate tool correctly. To be clear, asking for tool suggestions is the best way to find what tool you need unless the question isn't framed correctly.
So, when I see questions like “What tool does this generic-thing-I-need best?”, I know exactly what is going to happen next…
Few clichés are more worn out than the tired “ think outside the box ”. I still stay it, but when I do, I say it to literally mean do not conduct an analysis solely within the physical box (CPU). Remember, everything that happens with data has happened because a person or persons made it happen. People do not live in a box. They live and operate in the outside world.
People are behind actions .
Every bit of evidence you find has a reason to be there. Someone made it happen. There was a thought, a plan, an intention, and an action to make it happen. For evidence that should exist but does not exist, this lack of evidence carries the same weight since it takes someone to make it appear as if it did not happen .
Sometimes your job requires fixing a problem (such as a breach) and make it so the problem has a less risk of happening again. In many of these types of jobs, identifying the suspect would be a waste of resources since there is no remedy to the problem other than reducing the risk of the next intrusion. However, if you are in the business of catching bad guys, then you need to literally think out of the box with your forensic analysis.
Identifying the Modus operandi ( M.O.) can help identify the suspect’s intentions and identification. You can find the M.O. through forensic artifacts. You can also find more than just the M.O., like traces of evidence inside the box that lead to clues outside the box, such as geographical locations or the actual names of suspects. Outside the box, interviews with potential suspects, even those who may lie, can give clues as to what to look for inside the box. Just as it is important to validate your forensic findings, it is important to validate (corroborate) investigative findings with other facts. Finding evidence on a device is great, but it is much better when you have information obtained outside the exam that corroborates the evidence on the device.
The way I look at it, when tracking suspects, I use whatever clues I have at hand to lead to the next clue. If starting with digital forensics, that means I want to use that what I find in the box to lead me to a person outside the box. To do that, I need to remember that a forensic analysis is just a forensic analysis. But when you couple it with thinking outside the box, you get an investigation to find your bad guy, and not just data.
The new-peer-review-no-name-yet task force is chipping away at the proposal of a new (but extremely different) peer review process for DFIR research, spearheaded by Jessica Hyde .
I’ve gotten a few private messages that teeter on the edge of complaints about even talking about creating a new process of peer review, but each complaint has been relieved of worry after clarifying what we are working to come up with.
Here are some of the things I want to clarify:
Who should this process appeal to?
Who does this process not target?
So, you can see that this is primarily, if not solely, intended to give DFIR bloggers an avenue to have their work peer reviewed and be recognized for their work . It is so much easier to cite a peer reviewed work in other research, so the community benefits as well. The blogger benefits by having his or her name stamped with the research without fear that another person or commercial entity will claim credit for the work that was done. At the bare minimum for clarity, the work that will be peer reviewed in this process is work that would have never made it into an academic review anyway. That is the audience we hope to support: the DFIR practitioner bloggers.
The target audience is not expected to be earth-changing in size. Maybe just a few a year will have interest in the beginning. Plus, not every blog post is something that needs to be peer reviewed. But the research that is new, innovative, or creative…why not?
What a time to be in the field of DFIR! If you have being doing this work since the days of the floppy, you surely must be as excited as me. If you just entering the field, you will see even more advancements in the future than your predecessors have.
But let’s get on with one of the most important topics that is making our skill levels advance more than anything else has ever done before: Instant documentation and sharing.
Many in the field have written (and keep writing!) and about the importance of sharing and documentation. Without getting into ethical questions in the field about sharing special discoveries, I want to talk about sharing generically, but specifically in the physical manner of sharing.
One of the biggest issues in our industry is the dearth of documentation.— H. Carvey (@keydet89) July 2, 2018
The Internet gives us so many platforms to share information that it is practically impossible to keep up on it all. You cannot “follow” everyone. Google won’t find everything. Some platforms won’t give you access (only specific groups of people can access, such as LEO-only), and some platforms are simply too difficult to keep track of the information that flashes across the screen only to disappear into the “blackhole of great information but no one saw it”.
For the DFIR info curators, the DFIR blog is the number one source of information , mostly because it is semi-permanent, easy to find, easy to bookmark, and most always accessible to anyone with Internet access without having to have a special account to access.
Other means of dissemination are faster to put out and faster to reach an audience. Twitter is a prime example of being able to send out a bit of information in seconds that can potentially reach millions instantly. The negatives are that most tweets are not well-thought out, lacking depth, can be deleted, and are quickly buried in seconds by hundreds of newer tweets. On top of that, if you don’t follow the tweeter and no one that you follow retweets the wonderful information, it is as if it were never typed in the first place because you will never see it. In all likelihood, there have been outstanding tweets of information that were so quickly buried that few people even saw them.
Social media platforms like Facebook and Linkedin are only a little better in the sense that the posts seem to last a little longer, but still are not going to be as in-depth as a well-written blog post on research. Worse yet is that viewers need an account on most of these services in order to be able to see the posts.
In between the sites like Facebook and micro-blogs like Twitter, we have Discord , Slack , and other chat services. Again, you need to be a member of the group to access, the information in many of these services fly by the screen and is buried in a blink of an eye. And to even know about one of these services is to be lucky to catch the info on Twitter or be invited to the inner circle through contacts you have.
Based on research on social media content's lifespan from http://blog.hcpassociates.com/how-long-does-content-last/ , consider how the following graph relates to the DFIR information that we share.
Length of time content lasts on various platforms
The books and journals throw this graph out of whack since the content in a book or journal is measured in decades . But let’s take away the books/journals. Here is what we get for content lifespan.
Length of time content lasts on various platforms (minus books/journals)
So here you can see that a blog’s content remains for about 2 years , but still, other social media is not even registering on the chart. After this point of blogs, information doesn’t last longer than hours or minutes ( 18 minutes is Twitter’s lifespan! ). This chart doesn’t even include chat services like Discord, which I would imagine has a lifespan of way less than Twitter, maybe even lasting only seconds.
Add to this the closed lists, closed forums, and closed chats to get the real picture of how much information does not reach the practitioner or is stored with any permanence. Couple this with the amount of time that anyone has to keep up on dozens of services and you get a very dark picture of how much we know compared to how much we could know.
Therein lies the issue. The faster the information is disseminated, then faster it disappears, or worse, is never seen. The slower the information is made available, the more people that have access to it, but relevance begins to fade over time.
* Substantial/important information should be documented at least to the level of a blog post. The tweets and chats should be short bits of content with reference to the blog posts.
* Blog it on your blog
* Guest post it on other blogs if you don’t have your own blog
* Guest post it on other blogs even if you have your own blog
* Publish it formally
* Do the above, plus..
* Have it peer-reviewed and published
* Make a video about it
* You can do wonders with a short, 3-minute video
* You can embed the video in your blog post, tweet, and update when necessary
Time is always going to be an issue to research and share. Many of us barely have the time research, or fully dive into something unusual we come across in our daily duties. To require more than that is a lot to ask, but it is not unreasonable to ask to share bits and pieces as you can.
One thing I can advise, is that if you don’t share what you find, someone else will find it. And someone else may share it and clearly take credit for something you could have taken credit for. If credit is something that drives you, you need to put your name on it. If you don’t like discovering something and something else taking credit for it (when you never gave notice of your find), then you better share what you find. I’ve spoken to a few folks who have complained as if someone broke into their home and stole their research to publish, but in fact, who ever finds it and publishes it first is the person who discovered it, whatever ‘it’ happens to be.
As for me, these complaints fall on deaf ears. Put your name on it or someone else legitimately will (this goes for individuals and corporations).
I use Twitter in the event that I happen to come across something really hot that needs attention. But I also know that I miss 99% of everything that comes across on Twitter because I can’t live on Twitter. The same with Discord, closed forums, and such. The time it takes to log into a service and maneuver through it to find information is mostly time I don’t have. But I monitor blogs on a daily basis. There are hundreds of blog RSS feeds atdfir.training that I check daily (many times a day sometimes) to save time in clicking bookmarks to see who has updated what on which blog. Blogs last longer enough that I can check a few days later and not miss something. If you miss 10 seconds of Twitter, you will miss something. Phill Moore also saves me a lot of time with his blog which tends to catch things I missed.
So here's the point:
--Blog some more
--Tweet about your blog posts
--Let the blog curators (like me and Phill ) know about your blog to help it get traction
More on the Rapid Peer Review for DFIR blogs is coming. There is a special ops team led by Jessica Hyde on a mission to figure something out to benefit the community, the researcher, and just as important, the reviewers. One of the issues I see is that of ownership of current methods of publishing and peer reviews competing with several options of doing this differently. Mostly I dread someone taking a stance that their way is better, regardless if it is better or not even a competing issue at all. My personal goal is to help create a system in which those would not have published before, will be able to publish now, not to take away from anything that anyone else is doing. The point is to share, make it easy to share, and make it easy to use the shared information.
What started as a question on twitter, turned into a poll and twitter discussion, has begun to evolve into something interesting: The “ Rapid Peer Review ”.
I’ve had quite a few DMs and emails with several people over the past week on peer reviews in the DFIR world to discuss this topic.
In short, academic reviews take too long to publish and are of limited practical value for practitioners. We need a better system.
During these discussions, Jessica Hyde coined the “RAPID PEER REVIEW” name, so I’m sticking with that.
Since this idea is evolving, here are some of the ideas being discussed, all subject to change:
* Process should take 30 days or less to be considered Peer-reviewed or rejected
* Previously peer-reviewed work (as seen in a published journal) would be ineligible
* Previously written work that has been cited or referenced may be judged as already peer reviewed by virtue of being source material of peer reviewed work. Meaning, if you wrote something that was later used in books and journals, then your work was probably already peer reviewed by those authors of books and journals. This process would simply verfiy and validate your work as cited.
* Peer reviewers would be practitioners within high-tech organizations or academia
* Work that has been RAPID PEER REVIEWed could still be eligible for journal publishing (but not the other way around)
Here are the benefits to you :
* Your research is recognized .
* Your work get more exposure , not in the manner of becoming famous but instead your research is shared more widely.
* Your name gets credited for your work.
* The community has another source (a validated source!) for research to build upon and learn.
* Your ability to cite sources that have been peer reviewed increase, rather than citing someone's blog post (that is, if you cite for a book, journal, or legal matter that you are writing).
At this point, there are a few drivers pushing this idea along. I officially Knight Jessica as the cat herder. Eventually (sooner than later), after some details are fleshed out, it would be good to see a few more interested parties join in to help with the physical labor.
At any point, we encourage comments, suggestions, and recommendations at any time. Currently, we have some ideas on the who will peer review, where the peer reviewed documents will be stored, and how the process will tentatively work. After we whittle down a little more, I’ll write up the details of where it stands, which could look really good or need a total revamp…that’s where input will be helpful.
Who does RAPID PEER REVIEW affect?
One of the main points that the Rapid Peer Review process focuses on is that of avoiding the academia model at all costs. This is not to replace traditional publishing or scientific journals, or to compete with anyone who wants their research in a journal or book. This is for those who would not have published in a journal or book, or maybe is not ready to publish in a journal or book. This is for everyone who has written or will write some cool DFIR stuff that should be shared as a Peer Reviewed work (of art and science…).
What do you not get for having your work in the Rapid Peer Review process?
The intention is simply to be a bridge between a blog post and a scientific journal.
Brett's Peer Review Model
I have posted on this a few times, as well as commented on Twitter, but the short answer is: "We don't peer review because it is too much work and too much time spent with no real personal benefit." Our jobs are not publishing, but actually practicing the trade of DFIR.
Now I see another reason why DFIR researchers may not be publishing their work via the 'academic journals'.
I feel that DFIR had been doing it right all along. Practitioners work. They find something interesting. They blog about it. Then everyone else takes advantage of their discovery. And when it's really good, the practitioner writes up a Word doc, PDFs it, and uploads to the Internet. Now it is memorialized forever (or until the Internet dies). I had suggested that the DFIR community add one, little step between the PDFing and posting: community peer review. The reasons to add one thin layer of peer review is simply to validate the work that was done so that citing it becomes easier and the DFIR discoverer gets permanent credit. The community benefits overall.
After reading " Some science journals that claim to peer review papers do not do so ", I see that there are even more reasons to avoid the academic route to journals unless your job is in academics or you want to go into that field.
I agree that there is validation, credibility, and personal satisfaction in having an academic peer-review paper that is published in a journal, but everything that is required to do so goes against the very grain of DFIR work. DFIR research needs to be shared yesterday, not two years from tomorrow. The methods and artifacts that we discover are sometimes perishable, but certainly they are dynamic. The academic model for peer review doesn't work for DFIR research because it takes too long. In fact....very few practitioners read the scientific journals, and with that, the research will have been in vain.
My bias* as a practitioner is obvious, because there is no hurry in the academic world. The academic world does not deal with a breach where a business may go bankrupt in days, or where national security secrets are being siphoned out of a network, or where a child needs to be rescued after being lured online. Practitioners need the newest research as soon as it is ready ( ready to be put to use, not ready for the academic peer review process ).
As a matter of practicality, money probably needs to be involved in this process, because although I support working a job for fun, I do not agree that you should be required to work for free. How a business model needs to be developed for a non-academic peer review model is a topic that should be started sooner rather than later. The good news is that I see more than a few DFIRrs talking about it. Now that is cool.
*Side notes on my perspective and bias:
I have practiced DFIR in the public sector and private sector, and taught it in the academic world. I tend to see the importance of immediate access to research being more overriding in importance than a long-process of publishing.
Here is a brief list of reasons of why I think DFIRrs blog their research rather than formally publish it through a peer review process.
---faster (minutes to type up and post),
---easier (click “post”),
---written for the practitioner (“this is how you do it”),
---putting out perishable information before it spoils (“applies to the current OS today”).
--Peer review is:
---slower (months or years),
---more difficult process (lots of steps and hurdles),
---written academically (“for the love of all that is good and holy, get to the point!”),
---might be outdated by publishing date (“well, no one uses this OS anymore, but when they did…”).
Neither method results in direct a financial gain for the work done. The time spent will not equal money received, if any money received. No fame either…
I’m not going to get into the peer review process, as you can find plenty online. I will say that the process is long. Very long. Lots of steps. Lots of people involved. Requires lots of effort to check the boxes required for the process. I am including publishing research in a book as the time required is practically the same. I believe that good DFIR research should be peer reviewed, and that IF the academic model had a reasonable process and time frame for publishing, this would be the way to go. But that is not the case. Also, the writing methods of a journal are certainly not what practitioners want or need. I also believe that there is nothing wrong with a blog post having the most credible, up-to-date, and relevant information that the community can use instantly after the information has been posted to a blog. Nearly all DFIR blogs are written for the practitioner with clearly defined and described bullets on “This is how you do it”.
Back in da day , forensic folks would write up a pdf and put it on the Internet somewhere. I was one of them. These “papers” were basically blog posts that were put into PDF format to be easier to read and memorialize.
This paper that I published in 2008 was about virtualization and forensics . By publishing, I mean that I emailed my blog post as a PDF to ForensicFocus and they put it on their website. It took me a week or two to write this up after having been researching and playing with virtual machines years prior. I had sent a review copy to someone I knew for his opinion before sending it to Forensic Focus or posting to my blog. The response I got from my prof buddy was that I should put it out in a journal as I would be wasting the work by blogging about it instead of publishing it. Being a full-time professor, he strongly pushed the journal route. At the time, virtual forensics was not commonly practiced and I felt this to be an important topic that will eventually be part of most forensic analysis cases. So I emailed to it Forensic Focus instead and a day or two later, ForensicFocus posted it on its website.
The following week, or maybe the same week that this was posted to Forensic Focus, I read online that a book was going to be started on the same subject. There was a post asking for contributors to the book, so I emailed my paper and even offered to help with the book content (sadly, I wasn’t taken up on the offer…). The book was in print almost 2 years after my paper .
Between the time of my paper and when the book was published, my paper was referenced in multiple other papers (thesis’, other documents), quoted in at least one forensic book, and even quoted, referenced, and cited in the Virtualization and Forensics book. It’s been referenced over two dozen times total that I found online, but maybe even more that I didn’t find when searching online for a few minutes. These references to my paper are found in writings from several countries, both academic and practitioner. I sat in a course once where the instructor went over my paper as part of a lesson and didn't realize I was in the course (not that I cared, but it was neat). Don't take this as bragging, which something I don't do. Rather, I want to illustrate where this PDF "paper" went in a short period of time without being a journal or book.
I could have gone the academic route, but it likely would have taken a year from submission to print. On top of that, many journals are behind paywalls, or accessible only to an educational institution. Going even further, I wanted this information out a month before I even thought about writing it. Not for fame or fortune, but to share some really cool stuff about virtual machines and forensics. Today, virtual machines are not sexy news anymore, but back in those days, this was cool stuff that few people were doing that I thought should be aware.
Even going a little further, my paper furthered additional research in virtual forensics and ended up being cited in more than a handful of books and the same paper is still being referenced as recently as this year (2018), ten years after “publishing a paper as a PDF”.
In my opinion, this paper has met any peer review standard that exists, simply because of the places it has been referenced, quoted, and cited without any correction of fact.
Making this paper publicly available in two weeks had a wider and more positive impact to the DFIR community than it would have if I had chosen to publish academically as suggested by my professor friend. To be honest, that was my intention. Get the word out today, not tomorrow.
There are thousands of DFIR and InfoSec bloggers today. I have over 800 of them listed at dfir.training. That is truly a lot of information. To suggest that any percentage of them publish academically will flood the process and slow the release of information to the community. I put books in a separate category, because books are a little different in regards to perishable information. Writing a tech book requires not focusing on the perishable information and writing something that will withstand years of being relevant through concepts and principles that apply today and can apply tomorrow. That means much of a book’s information is known somewhat, but hasn’t been put together as a package. Not so much with blogs. Blog information is sometimes perishable because technology changes faster than what it takes to print a book.
Also, anyone working in this field is short of time. We are busy with work. We are busy with keeping up enough to do a good job with work. We are busy with family. We are just plain busy and to add more to our lives is asking a lot if there isn't a mutual benefit.
When we write up something important (or something cool..), we want to share it. Blogging is the fastest method* and I would argue, the most effective method to disseminate DFIR methods, processes, and discoveries. Peer reviewed journals do not further the field if the information in the journal is not shared immediately after being validated, because by the time a journal or book is published, the information may be stale, outdated, or has become commonly known. Not in all cases, but certainly in many.
Create a NEW process that combines the best parts of both peer review and instant blogging.
Cut out chunks of time needed in the peer review process. Allow the work to also be published by any and all means that gets the information out. This includes blogs, forums, chat rooms, and courses. Nothing wrong with the information later being used for a journaled piece or expanded into a book but get the information out now.
Who peer reviews in this new process?
I would recommend that academia and high-tech associations be involved in the peer reviews. The system is already in place. Use it by modifying it for speed. What possibilities will be created if someone who wrote up a substantial finding could submit it to the local high-tech group or local university for a peer review and stamp of approval? We could have blogged information, peer reviewed by credible organizations, out faster than any journal or book.
Deciding upon a process, a standardized paper format, and types of papers to be considered is not that difficult. Give this process to any participating DFIR association and educational institution. Peer reviewed by one should equal the weight peer reviewed by another, as long as the same process was used.
And what of the stuff that is so cool, I mean really BIG, that it should be in a journal or book? Then the organizations can suggest or assist in that process while still getting out a practitioner-level, peer-reviewed paper in a timely basis and put out the formal journaled paper later.
Did you find something important to the community? Why not stamp your name on it with a peer-reviewed paper? If the process is fast and pain-free, there isn’t any reason to not do it, unless you just don’t want to. Even if it barely covers a page or two, why not make it formalized to make sure it gets traction and more importantly, is made part of a permanent record, with your name on it.
This will take more than a few folks to implement. Everyone will have to donate time and effort to just get it off the ground. Everyone will have to believe it too. If you ever considered publishing your work, but choose not to because of the process, this is for you. If you ever did something cool but it didn’t meet the requirements of a journal, this is for you. If you are a board member of a high-tech association or in DFIR academics, this is right up your alley. You can make a substantial impact to the DFIR community with something like this.
I know that either we will make the system better with publishing our work outside of a journal but not to the extent of a book, or we will accept what we have . We will continue to have blogs that disappear after a few years (along with the information that was on it), and stilted journals that should have been blogged instead of going through a year long peer review journal process.
As for me, I’ll keep blogging, PDF’ing, and writing books. But I hope that a new process can be created to change with the times to help all of us keep up with DFIR.
*Peer reviewed correction by Phill Moore
Twitter is faster than blogs but for the love of everything that is holy please document things in a blogpost as well. Tweet streams are just annoying— Phill Moore (@phillmoore) June 20, 2018
Following up on a forensic artifact project database idea , the end result is that the idea is dead before it started.
The twitter poll (one of the most unscientific, but easiest polls to do) didn’t show a lot of promise. Also, there were a LOT of DMs and email discussions. Thanks to everyone giving me their thoughts.
Here are the main points that I received, summarized in three statements:
-Publishing research must be in academia (journals)
-Publishing research must be in books (publishers)
-We don’t need project management in research
On top of these points, the fear of lack of contributors holds me back. According to the Twitter poll, less than half (of only 88 who voted), would contribute. That is not quite a big enough number in percentage or actual number given how many people actually do research and compared to the discussions I had with some very passionate folks.
Some suggestions given were;
-Those who have published and those who have done research should connect with each other to publish the research.
-Those who do research should go through the academia route to publish in journals.
I don’t see this happening to any great degree, other than a perhaps a handful of instances.
We are left to relying on DFIR/Infosec bloggers for the most current research, which will have to make due for citable sources that do not exist in publications. Ironically, this was the original tweet concept that started the conversation to begin with.
As to immutable citation sources, we still have books and journals, and everything else will be dynamically changing and evolving, which is a double-edged sword. Good in the sense that we have nearly instant access to the newest developments via blogging. Bad in that blogs are not peer-reviewed, nor immutable. Blog content changes, which can make for a confusing citation. Blogs also disappear without notice, which again, affects citations.
I do foresee a time where a practitioner will be able to more quickly publish a peer-reviewed and community accepted work outside of pure academia, but unfortunately, it is not today. The peer-review process, as it stands in academia, is a long process and requires probably more time to finally be published than it did for the actual research. This should be opposite, but it is what it is. Most importantly, however, is that the DFIR/Infosec blogs are awesome for the most up-to-date, practical, and useful research that exists on the planet. Do not discount any research that was personally conducted by a practitioner. It may be right. It may be wrong. Regardless, each is a nugget of gold to expand upon and personally validate in your own research. For that, if you are a DFIR/Infosec blogger, you have my respect.
Thanks to all who contributed their opinions!
A weekend Twitter thread about having a lack of citable, peer-reviewed, DFIR research prompted me to volunteer to host a project management website (a sub-domain of dfir.training). I think the need is real for reasons mentioned on the Twitter thread, but whether or not it can work is all together a different matter.
From what I have seen, peer reviewed DFIR research generally lives within journals and books , or within the walls of academia . Either the research is not freely available and/or not easily found within the walls and halls of educational institutions. Research is blogged about, presented at conferences, and uploaded to the Internet via any number of websites, with much of this work not being peer reviewed . There is too much great effort that is never formally published in which the researcher deserves to (1) receive formal recognition and (2) be formally peer-reviewed by the community without having to be published in a journal or book.
I absolutely give credit to the bloggers sharing their research online, more than you can imagine and for many reasons. However, referencing a blog in a case report, affidavit, or in a research paper does not quite reach the level of peer reviewed research as a source of information. The life of a blog is also indefinite, dynamic in nature, and many times never found by those who need the information.
Given enough interest, I will gladly maintain the website, and ensure that the research will always be freely available. I will manage the users and research groups to reduce the risk of trouble-makers (such as bots and spammers) from having access to the editable parts of the research projects.
Playing devil’s advocate, here is what I see:
-Lack of willing contributors
-Lack of willing peer-reviewers
-Continued reliance on wiki’s and blogs as a source of non-peer reviewed research
-Continued non-sharing of personal research
Countering this, I see:
-Contributors being those who have already completed personal research, who can now have their work peer-reviewed
-Peer reviewers growing professionally by helping and mentoring researchers
-Reliance on credible, peer-reviewed, professionally monitored research for citable resources
- You personally being credited and formally validated by the community for your work
-Research that is developed, peer-reviewed, and published months faster than in a journal or book
-Research that meets Daubert-Frye standards (community accepted methods)
- You get another source of community validated research without having to pay for it
I have a Twitter poll that will expire in a few days. If not enough interest, perhaps the timing is not right. Personally, I think it is past time for an additional means to create peer reviewed research for those who would rather see their efforts received by the community-at-large, rather than kept behind paywalls or within privately accessible collections of research.
As to the mechanics of how this can work
-Project manager (researcher) initiates a research topic
* Project created
* Tasks created
*Contributors join in the project
*Project manager keeps the research going through final draft
*Public does not have access to the active research (unless they want to contribute)
-Final drafts reviewed by peer review process (any number of types, such as a blind review, double-blind, etc…)
*Corrections, suggestions, recommendations made
*Sent back to project team
-Project team makes corrections as necessary (or defends any claims against research)
-Final peer review and publish (accessible to the public)
All of the above is through a project management platform. The final peer reviewed and approved project would be in a standard format. This concept is to provide an additional means of peer review for that research which currently sits on websites without earning documented community credibility.
Imagine taking research you have done in the past, perhaps it's on your blog, and you get it peer-reviewed, meets Daubert-Frye standards, and becomes available to everyone as a credible source of DFIR information. That's the point to all of this.
Or, we can stick to citing blogs and wikipedias…
I was speaking to someone at Infosec Europe last week about ‘getting into this field of infosec’. I kept answering all questions with the same answer of telling the guy to get started and do something. But the future DFIR’r kept telling me about all the training and schooling that he had completed, the training and schooling that he is planning to do, and what to do next. I was quite impressed with how much training already done, including earning a degree and having taken a dozen vendor courses. I was disappointed in how much more he was planning to do before ever starting work in the field that he has spent years in learning, but not doing.
In short, I told him to stop his training and education right now and make this conference his last until he puts to work what he has learned so far.
He was stuck in learning mode, repeating courses and conferences over and over again with the expectation that competence will come to him. This is a terrible mistake. There is a point where you have enough training and education to start, where you can do the most basic of job tasks, and where you can apply what you already know to learn more through experience. To be stuck in learning mode is to never know what you can through practical applications. Testing, theories, and essays are only part of the equation in becoming competent.
Don’t get stuck in learning mode. This applies even if you are competent in your field with experience. At some point, which is different for everyone, spending more time in education isn’t going to propel you farther than if you are actually doing the work, practicing that what you know, and discovering what is not being taught in class. Avoid the point of diminishing returns on “learning” when you have more than enough to be “applying” what you have already learned.
I’m not advocating to never attend a conference or training course, or not to get an advanced degree. And I am not saying that experience is better than education. I am saying that there is a balance needed between education and experience. Having a balanced portfolio of experience, education, and continuing learning builds your competence base much better than focusing solely on the academic or solely on the experience aspect of DFIR or any field for that matter.
When I say, “Get out of the learning mode!” , I am saying that in the manner of take what you have learned formally and put it to use physically. You will still be learning, but you will be learning differently, and learning things you won’t learn in a formal training atmosphere. You will learn by doing, which will make your future formal learning that much more effective because you have had your hands on the things being talked about in a classroom.
I used to sarcastically joke in my police days that some cops seem to be in training every other week and had taken so many training courses that the only training left was underwater handcuffing classes. Training and formal education, much like training wheels on a bicycle, is to get you up to speed to start working on your own where you can excel well past what any training wheels can do for you.
Break the Groundhog Day cycle. Put your knowledge to work and complete the path to competence by learning AND doing.
Like many others working in DFIR, I occasionally get asked questions on how to get a job in DFIR. By DFIR, I mean the overall field of digital forensics/incident response/electronic discovery. Sometimes, the questions are loosely asked as if it is easy to get in by someone who thinks they are 'good with computers'. Other times, I am asked by those with computer science graduate degrees and tons of computer experience. The range is quite wide. I am certain that anyone and everyone looking to make a break into DFIR has already Google'd it, found a lot of blog posts, and still are having a difficult time getting in the door. That is just the way it is. Employers feel like they can't find anyone and everyone feels like they can't find an employer to hire them.
I have blogged about this before, and I'm writing again because this is not only a common topic, but it is really important if you are trying to get a job in DFIR.
I wrote a short paper on my thoughts, ideas, and opinions that you can download by subscribing to the DFIR Training newsletter . Yes, I am sneaking in the newsletter to get the download, but you can always unsubscribe if you don't like what I do with the monthly newsletters. I am betting that you will get a benefit of the newsletter.
As far as to giving another opinion on how to get into DFIR, I believe that the more people that give their opinion and experiences on how they got in, the more likely that someone will be able to use that guidance. We each have had different paths to "make it". Some of us had no help, others had a little help, and a few fortunate few had a lot of help. But everyone who is here can easily lend a hand to the new folks coming up by giving a few words of positive advice.
Oh yeah, I created a "Get Hired for DFIR" checklist on the download for those who like checklist. I am one of those who use checklists, which is why I made the checklist :)
For those working to get into DFIR, you can do it and we are waiting for you to check the boxes and get hired!
All right. You got me. I am not going to force you to subscribe to get a download. So you can download the file here: Unlocking the DFIR Door . But I think you should subscribe anyway. The newsletter will be awesome!
I am starting a monthly newsletter to supplement the existing newsletters that many other DFIR contributors are creating. I intend to make the newsletter different enough to justify having one email a month in your inbox; actually, I intend to create an awesome monthly newsletter.
I initially expected to give this a shot for a month or so and then see if the signups would justify the effort to create a newsletter. MailChimp has a ‘forever free’ plan that looks to fit what I’m looking for so for testing the water for interest, MailChimp it is then. However, after just a week, the signups are reaching the limit of the free plan. No biggie as this is a pleasant surprise to see the interest. The first newsletter won’t come out until June, but you can sign up now so that you don’t miss it.
A DFIR Training Helper!
One of the things happening at DFIR Training is me getting a helper! This should make updates to the website go a little faster (faster additions of artifacts, tools, etc…). Plus, I will have time to catch up on the projects that keep getting pushed farther and farther behind.
Upcoming site features
There will be an opportunity for guest blogging at DFIR Training. The purpose will be to give traction to anyone wanting their research to be seen by more people. My suggestion is that if you have a blog that you want help marketing, either write a unique post or cut-n-paste something from your blog for a guest post (and link DFIR Training to your blog). The goal is more exposure for research and more credit for the work you do.
Other website additions are coming as well, which I’ll keep under wraps until I actually start them once my helper gets more fully on board. Expect to see uniquely created content soon enough.
DFIR Training is popular because…
….it has what you want on it. Visitor stats are high and the site ranking has trended upward fairly well. Page views and data transfer is astronomical ( 90GB of bandwidth and 4.5 million hits in 2017). I suspect this to go higher as soon as I start adding the new stuff over the summer.
I put up more stats here if you are interested: About DFIR Training
What else do you want to see?
If there is something you want on DFIR Training, send me a note and let me know. Complaints, compliments, and suggestions are welcome if the intention is to make the site better for the community. In other words, let’s be nice ?
It has a lot of DFIR stuff
It really has a lot of DFIR stuff
Even more DFIR stuff is coming!
I checked the stats for dfir.training for the months of April 2016, April 2017, and April 2018 to get a gauge on what pages are most popular, where most visitors are coming from, and areas to focus on content (based on the pages and page behavior flow). What I found is that the stats have dramatically increased since April 2016, particularly toward the end of 2017 (The stats do not include bot traffic, which would unnaturally inflate the hits). My conclusion is that so far, everything seems to be in order since the number of visits and hits are higher than ever before.
Even tho the visit rate is high, I am still planning to add more content (new stuff!) that I have been thinking about but haven't had the time to do yet. The new stuff is in addition to the regular updates and content that needs to be added incrementally, like the forensic artifacts and tools. I will have a website 'helper' later this year so that the amount of updates and data entry will be twice as fast as it is now.
With that, and considering the number of visits.......if you are one of the dfir.training visitors and you find the site useful, your vote for DFIR Training as Digital Forensic Resource of the Year would be appreciated.
Hurry as the last day to vote is coming up quick (May 25).
As to the stats, here are a few stats from April 2016, 2017, and 2018.
I visited a DFIR shop and as I was leaving, I asked one of the most experienced examiners in the shop, “ Hey, how do you like *x* tool new functions ?” and the answer was “ Never heard of *x* tool .” For me, I use *x* tool often and assumed that everyone else does, or at least knows about it. I was wrong. (I am not naming the *x*, but it can apply to any tool you use). If you don’t know that a tool exists, you are not going to use it.
The DFIR.training tool database
My opinion is that looking for a DFIR tool that does a specific thing that you need for a specific analysis is either easy or impossible, depending on how you look at it. If you only use the major name brands, you have an easy choice (because you only want to use a major name brand for everything). If you are looking for a very specific tool to do a very specific thing, you may have no choice because of being unable to find it. So, the DFIR.training tool list contains a lot, because maybe that one little tool you need might be there, sitting under the category you are looking.
Top 10 Lists
Often times, I find online lists of forensic software that are ‘best’ for your lab. You don’t have to look far to find lists that are something to the effect of the top ten free tools or the ten tools that you must have in your forensic toolbox. You can find suggestions, opinions, and even detailed scientific methods on how to choose a forensic tool.
Some warnings on pre-defined tool lists.
-They are not personalized to you. Others created the lists, based on their opinion and needs, or what they think you need.
-Lists are limited. A list with “The Top 10..” may not fit your needs, but maybe #11 would, if it were on the list. Maybe #33 would be #1 for your needs, but #33 won’t be on someone else’s Top 10 list.
-Lists can be irrelevant. How can you compare RegRipper with EnCase on the same list? Dissimilar tools compared with each other makes for an irrelevant list, but it is common to see.
When I see a list, I only look to see if there is a tool that I never heard of before, not that I accept it to be put on my personal Top 10 list because someone else says it should be.
If you don’t know that a tool exists, you won’t ever use it, even if it would be perfect for your needs. I have found some gold in Github on more than one occasion, simply by searching Github in hopes of finding something that I need but can’t find elsewhere. This takes a lot of time, but you can’t expect that a tool on Github has a marketing budget to get the word out. You have to search for it..if you have time.
Commercial companies have marketing budgets and marketing operations. If they did not market their products, few would ever hear about them, fewer would purchase them, and eventually that company closes down. This is a loss for everyone. Word of mouth only goes so far. Consider that I found a few tools in Github that could be huge commercial successes, but without substantial marketing, won’t make it into mainstream DFIR. Can you imagine if no commercial tool was ever marketed? Where would you be right now with your tools if there was no marketing?
The Menu Method of Finding the Best DFIR Tool
Rather than a scientific method, or picking from a pre-defined list, I look at DFIR tools like I look at a menu. If I am looking for breakfast, I look first at the breakfast menu for something that I may want. Sometimes, I might choose from the lunch menu for breakfast because the lunch menu is what I want at that particular time. The menu is simply an offering of things from which to choose. Just as one food can be served at breakfast or dinner, one DFIR tool can be used for collection or analysis. The menu is simply a guide.
If a particular menu doesn’t have what I want, I go to another restaurant and look at a different menu. And if none of the menus have what I want, I have to learn to cook it myself. The menu options in DFIR are your categories. Some tools fit neatly in one category, others fit in several categories.
Your Top 10 DFIR Tool List
Looking for DFIR tools work the same way as deciding what to eat for lunch. You consider your wants and needs to make a decision. I am sure that you don’t decide what you want for lunch for the rest of the year. You probably make a different decision, or at least go through the same decision-making process every day, because every day is different.
DFIR tool selection works the same. Every case is like deciding what to eat for your next meal. What kind of case is it? What do I need to do with the case? What tools will best do that job in the case? Which tools do I prefer and do these tools match with the best tools for the job? Unless you are tied to a specific tool for some reason, the choice is yours to make and not someone else’s.