How to restore deleted knowledge in OpenCTI 6.1

Mistakes are inevitable, and until version 6.1, deletion actions were not reversible in OpenCTI. Who hasn’t felt the frustration of accidentally deleting APT28 or Cobalt Strike from the platform, with their countless relationships?

In this article, we will describe how OpenCTI now handles deleted knowledge and makes possible restoring or permanently deleting data.

---

Introducing the new trash view

Since OpenCTI 6.1, when you delete a knowledge object from the platform, whether it be an entity or a relationship, the object is re-indexed in the Elastic database to a dedicated “trash” index. The deleted object is saved along with the relationships that originate from or target this object, before their deletion.

The deleted objects are listed in the new Trash view, accessible from the left menu. In this table, you can search, filter and sort the deleted items the same way as for any other view.

However, note that the items listed in the Trash view are not the real data objects: you cannot access them in full to see the description, the associated content or knowledge. This table actually lists the Delete Operations that were made on the platform.

The delete operations listed in the trash view share the same access restrictions as the object that was initially deleted: markings, organization sharing, confidence level. Thus a user with TLP;GREEN maximum marking will not see if a deleted object with marking TLP;RED is in the trash. Other metadata are also saved to track the deletion date and the user who deleted the object.

Technical details about deletion

It’s important to note that deletion action didn’t change: it will still publish a delete event in the stream, and the deleted elements are no longer visible on the platform. If two OpenCTI platforms are synchronized including deletion, the deleted elements will be removed from the two instances and will not appear in the remote instance’s trash.

Note, however that the files associated to the objects are not deleted from the storage service (S3 / Minio), but are marked as removed from the file indexing and will not come out during a file search if file indexing is enabled. These files are kept in case of restore, and will be permanently deleted once the trash element is cleared.

Restoring knowledge

From the trash view, you can choose to restore elements individually from the “dot” menu. This single restore action includes the whole deleted object and all its relationships.

The restore action requires manual confirmation. Restoring a very large number of relationships might take some time.

After confirmation, the object is restored, and all the core relationships associated to it are recreated. The internal identifiers of the object and relationships are preserved, so we do not lose any information originally linked to it, such as their activity logs.

It’s important to note that the object and its core relationships are recreated one after the other, and thus these operations will appear in the event stream as creations, triggering any configured automated actions such as playbooks.

Simple restore

Restore is currently possible only if the deleted object is not found in the database (no duplicate), and if its relationships can fully be restored with their targets resolved.

The main use case is when someone deletes an object accidentally, goes to the trash and restores it right away. If in the meantime the object has been recreated in the database (with the same name or identifier), restore won’t be possible since it might create a duplicate.

If the deleted object, let’s call it A, had a relationship with another object B, and A is deleted, the relationship with B is also deleted along with A. If A is restored, it will restore also the relationship with B. But, if B is deleted before restoring A, it won’t be possible to restore the relationship unless B is restored first. In that case A fails to be restored, as we do not handle partial nor cascade restore.

Permanent delete

For some entities that are in the trash, you might decide that you are sure about their removal from the platform and want to get rid of them definitively.

If that is the case, you can use the Delete permanently action to remove the element from the trash. The object will then be completely deleted, along with its relationships and files, and the action cannot be undone. So make sure you are absolutely certain about the item deletion before doing so!

When you permanently delete the object, the object and its relationships are removed from the trash index in Elastic database, and all files uploaded under this object (import, export) are definitely removed from storage. The delete operation is also removed from the trash list.

If you do not want to handle the permanent deletion of elements from the trash manually, don’t worry! A Trash manager, that will be introduced to you further down this article, is doing that job for you.

Bulk operations

In addition to being able to restore or permanently delete items in the trash one by one, we have added the ability to do these operations on multiple items at once.

By selecting multiple items the same way it can be done in the other listing views in the platforms, you will have both the restore and the permanent delete operations available from the bottom toolbar. When choosing one of those two options, you will then be able to apply the operation to all the items you selected.

Note that when deleting objects in batch, for instance from the Data > Entities view, all items will be listed individually in the Trash view as separate delete operations.

Trash manager

To prevent trash items from piling up and being stored indefinitely, we have added a Trash manager to the platform.

The goal of this manager is to permanently delete items that have been stored in the trash for too long. The threshold at which an item is considered too old is set to 7 days by default and can be configured using GARBAGE_COLLECTION_MANAGER__DELETED_RETENTION_DAYS environment variable.

The full configuration details for this manager can be found in the documentation here: https://docs.opencti.io/latest/deployment/configuration/#engines-schedules-and-managers

Conclusion

Mistakes are inevitable, but your knowledge base is now more resilient! You have just deleted an entity or a relationship by mistake? Find it easily in the trash view and restore it!

This simple — yet potentially catastrophic! — use case has been our main focus when designing this new feature in OpenCTI 6.1. Significant improvements might be implemented to increase resilience even more and address more complex cases, such as automatic cascading restore or allowing partial restoration.

Join us on our Slack community channel to give your feedback and help us shape the next move!

Checkout the user documentation for complementary information.