VMD-L Mailing List

From: Vermaas, Josh (vermaasj_at_msu.edu)
Date: Thu Sep 02 2021 - 07:46:55 CDT

Hi Bart,

Anything is possible with enough engineering, its just that reading a numpy array directly from tcl code by implementing a npy file reading proc likely requires too much engineering to be a good idea. The least-cost option for making the files more manageable is likely to gzip the files as they are written, and then use some of tcl's inbuilt gzip reading utilities to uncompress the files on the fly (https://urldefense.com/v3/__https://stackoverflow.com/questions/23384527/how-can-i-read-a-gzipped-file-in-tcl__;!!DZ3fjg!oex1p6Q4X9FcaJ3KzLyLVFL8Yp9TJktH1A5dTwtZYt59Yr1rx-0phf-thdS-xHfU6g$ ). You might also consider tclpy, a library that lets you call python functions from tcl (https://urldefense.com/v3/__https://wiki.tcl-lang.org/page/Experimenting*with*numPy__;Kys!!DZ3fjg!oex1p6Q4X9FcaJ3KzLyLVFL8Yp9TJktH1A5dTwtZYt59Yr1rx-0phf-thdSZwe8R5A$ ). In short, this is why I made the decision long ago to learn how to compile VMD with python, since that was for me the path with the lowest cost.

-Josh

On 9/2/21 5:15 AM, Bruininks, B.M.H. wrote:
Hi Josh and VMDers,

Thanks for the quick reply. I was thinking something like this would be possible, sadly enough the readable file writing will quickly become impossible. The software is meant to bring order in simulations of sizes which actually show complex morphologies and transitions between them (starting around 0.1M beads and upwards). In the start of the development we indeed wrote readable files, but it was not sustainable. If I read your answer correctly it doesn't state that reading a binary is impossible :p Or is this more a nice way to phrase practically it is impossible to do so via TCL. Again thanks for the reply and I think I will add this anyway, for something is better than nothing. However, the file size problem is very real.

Cheers,

Bart

On Wed, Sep 1, 2021 at 5:51 PM Vermaas, Josh <vermaasj_at_msu.edu<mailto:vermaasj_at_msu.edu>> wrote:
Hi Bart,

It looks like you need some way of taking your per-frame cluster identifiers and storing them into the user field. Lets say that you wrote a converter to save the contents of clusters_ordered.npy as a set of text files (clusters_ordered_0.txt, clusters_ordered_1.txt, etc), so that each cluster identifier is stored per frame. Assuming you had a proc to read in those files and return a cluster assignment for every atom in your atomselection (I’m going to call this proc “readfile”), the tcl loop is actually not that crazy.

set asel [atomselect top “all”]
for { set f 0 } { $f < [molinfo top get numframes] } { incr f } {
$asel frame $f
#Maybe do a file existence check here? That would be equivalent to your try block
set clusterdata [readfile [format “clustered_ordered_%d.txt” $f]]
$asel set user $clusterdata
}

So conceptually this isn’t too bad. You’d write a python script that takes the npy data and makes it into something you can write a tcl parser for. The files are going to be huge, since tcl and binary files are a pain, but it will prevent the end user from having to compile VMD+python support.

-Josh

From: <owner-vmd-l_at_ks.uiuc.edu<mailto:owner-vmd-l_at_ks.uiuc.edu>> on behalf of "Bruininks, B.M.H." <b.m.h.bruininks_at_rug.nl<mailto:b.m.h.bruininks_at_rug.nl>>
Date: Wednesday, September 1, 2021 at 10:31 AM
To: VMD Mailing LIst <vmd-l_at_ks.uiuc.edu<mailto:vmd-l_at_ks.uiuc.edu>>
Subject: vmd-l: Visualizing segmentation/labeling without python dependency

Dear VMDers,

A while ago I asked for help using python and VMD to visualize dynamic segmentation of molecular systems<https://urldefense.com/v3/__https:/github.com/marrink-lab/MDVoxelSegmentation__;!!DZ3fjg!u97BRTLa62glZFwoN6WosdIloozOsFY76535byxJxhMjfl03FNReD4ud4jpzewiNBQ$>. A script came out of it and it is working like a charm! Thanks for that again. However, it still requires a special compilation for VMD which is not really approachable for many users. As the segmentation is working very well now and the paper has been accepted, I was hoping we could maybe circumvent the python dependency completely.

During development our output for segmentation was saved as an npz file. However, we can easily change this to a type which is support by VMD. Hopefully then the reading of dynamic segmentation data can be achieved completely through TCL and allows reading of such data using the default VMD version. It would be great if I could get in contact with a developer and work on the TCL script. I think the answer must not be difficult, I just feel like I do not see it.

Cheers and thanks in advance,

Bart

The current script we have is:
# -*- coding: utf-8 -*-
"""
Created on Mon Jul 8 12:38:38 2019
This file is used to run within VMD with python2 and numpy compiled.
After succesfully running this file you can used the 'user' selection
identiyfer in VMD followed by a cluster number to visualize that cluster.
You can save the visualization state after having the right settings in
vmd, but you will always have to excecute this file first before loading
the visualization state (you should remove the load .gro and .xtc from
the VMD statefile).
@author: bart
"""
from Molecule import Molecule
from atomsel import *
# needed for opening the array.npy (possibly compressed)
import numpy as np

mol = Molecule()
# The .gro file should have the same order as the tpr used for clustering.
mol.load("../md.gro")
mol.delFrame() # Reading in a .gro file adds coordinates, which you don't want.
mol.load("../all.xtc") # This should be the same .xtc used for clustering.

# The cluster file generated to visualize in vmd.
clusters = np.load('clusters_ordered.npy')
asel = atomsel("all")

for frame, cluster in enumerate(clusters):
    try:
        asel.frame = frame
    except IndexError:
        print ('Not all frames could have their clusters assigned. Stopped in \
frame ' + str(frame) + '.')
        break
    print cluster, len(asel), len(cluster), frame
    asel.set('user', cluster) 

--
Josh Vermaas
Assistant Professor, MSU-DOE Plant Research Lab and Department of Biochemisty and Molecular Biology
vermaasj_at_msu.edu<mailto:vermaasj_at_msu.edu>
https://urldefense.com/v3/__https://prl.natsci.msu.edu/people/faculty/josh-vermaas/__;!!DZ3fjg!oex1p6Q4X9FcaJ3KzLyLVFL8Yp9TJktH1A5dTwtZYt59Yr1rx-0phf-thdRn0LUC9w$