[Gephi-plugins] Plugin for realtime graph generation based on web browsing

Mathieu Bastian mathieu.bastian at gmail.com
Mon Dec 20 04:04:11 CET 2010


Hello

1) When using the container method, is it possible to not have the
> coordinates of nodes change when they are referenced repeatedly?  In
> general, the node that represents the client gets a new edge on just about
> every update.  Each time a new edge is added, the node is repositioned on
> the graph to a new location.
>

> 2) On a similar note, is it possible to incrementally update the attributes
> with new data?  It seems that each time a node is referenced, the attributes
> are replaced, and their old values lost.  It would be nice to store that
> state in the graph and not maintain separate data structures in the plugin.
>
>
You are using the AppendProcessor I guess? It could be improved to support
these two points I think.


> Here are the links again:
>
>
> Code available at: lp:~phreakocious/gephi/httpgraph
> NBM at: http://phreakocious.net/dev/net-phreakocious-httpgraph.nbm
>
>
>
> On Fri, Dec 10, 2010 at 10:21 AM, T B <phreakocious at gmail.com> wrote:
>
>> Sure.. The largest nodes represent clients.  If you are the only one
>> pointing at the proxy, you'll only have one client node.  The smallest nodes
>> represent resources (aka URLs or URIs) (html/css, images, etc.)
>>
>> When your browser makes a request, a URI node is created for each
>> resource, and an edge drawn from the client to that resource.  Also, if the
>> request specifies a Referring page, a URI node is created for the referer
>> and an edge drawn from the referer to that resource.  This allows the
>> linking structure between pages to be revealed.
>>
>> Host nodes are created for the hosts that serve the resources, with the
>> appropriate edges, and finally, domain nodes tie the hosts together.
>>
>> Each domain gets a color (up to 32) and the other nodes should get the
>> domain color.
>>
>> NBM updated with some more bug fixes:
>> http://phreakocious.net/dev/net-phreakocious-httpgraph.nbm
>>
>> Enjoiez!
>>
>>
>> On Fri, Dec 10, 2010 at 3:23 AM, Mathieu Bastian <
>> mathieu.bastian at gmail.com> wrote:
>>
>>> Cool I tested it and its working good :-)
>>>
>>> Some questions,
>>> - could you explain what are the differents nodes, some of them have
>>> URLs, some not
>>> - what represent a link between two nodes
>>> - does the node color mean something?
>>>
>>>
>>> On Wed, Dec 8, 2010 at 8:25 PM, T B <phreakocious at gmail.com> wrote:
>>>
>>>> Switched from LittleProxy to Membrane:
>>>> http://www.membrane-soa.org/soap-monitor/  Haven't had any issues..
>>>>
>>>> Launchpad updated, new NBM at
>>>> http://phreakocious.net/dev/net-phreakocious-httpgraph.nbm
>>>>
>>>> Looking forward to hearing any feedback.
>>>>
>>>>
>>>>
>>>> On Tue, Dec 7, 2010 at 11:28 PM, T B <phreakocious at gmail.com> wrote:
>>>>
>>>>> Thanks, it has been stretching my Java abilities a bit.  I wasn't able
>>>>> to rewrite it with Import as you suggest, but was able to get something
>>>>> mostly functional from the advice.  This is still a Generator, but it stays
>>>>> alive in the background updating the workspace until the app is closed.  All
>>>>> you have to do is (install plugin), pick New Project,
>>>>> File/Generate/HttpGraph, set your browser's proxy for 127.0.0.0:8088(or whatever port), and browse the web.
>>>>>
>>>>> Some issues:
>>>>> Strange nodes with incomplete information
>>>>> Nodes without edges??
>>>>> Highly complex web sites (Google maps, for example) will bring
>>>>> everything to a grinding halt
>>>>>
>>>>> Code available at: lp:~phreakocious/gephi/httpgraph
>>>>> NBM at: http://phreakocious.net/dev/net-phreakocious-httpgraph.nbm
>>>>>
>>>>> Merci beaucoup!
>>>>>
>>>>>
>>>>>
>>>>> On Mon, Dec 6, 2010 at 11:43 PM, Mathieu Bastian <
>>>>> mathieu.bastian at gmail.com> wrote:
>>>>>
>>>>>> That is cool stuff!
>>>>>>
>>>>>> Yeah Generator is not the best place for doing that. I would recommend
>>>>>> to use Import directly, but woithout UI, to discard displaying the import
>>>>>> report. As you were trying to do this with Generator you are probably
>>>>>> already familiar with the container. Here is how to create, fill and append
>>>>>> a container into the current workspace.
>>>>>>
>>>>>> //Get Currennt workspace
>>>>>> ProjectController pc =
>>>>>> Lookup.getDefault().lookup(ProjectController.class);
>>>>>> Workspace workspace = pc.getCurrentWorkspace();
>>>>>>
>>>>>> //Get container
>>>>>> Container container =
>>>>>> Lookup.getDefault().lookup(ContainerFactory.class).newContainer();
>>>>>>
>>>>>> //Append container to current graph
>>>>>> ImportController importController =
>>>>>> Lookup.getDefault().lookup(ImportController.class);
>>>>>> importController.process(container, new AppendProcessor(), workspace);
>>>>>>
>>>>>> Mathieu
>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Mon, Dec 6, 2010 at 1:29 PM, T B <phreakocious at gmail.com> wrote:
>>>>>>>
>>>>>>>> Gephi devs,
>>>>>>>>
>>>>>>>> The idea is to stream data about your web browsing directly into
>>>>>>>> Gephi in real-time.
>>>>>>>>
>>>>>>>> After successfully proving out the concept with a Perl script and
>>>>>>>> the Streaming plugin, I'm trying to implement it in Java.  I've embedded
>>>>>>>> LittleProxy (http://www.littleshoot.org/littleproxy/) and extended
>>>>>>>> a few classes to collect the necessary data and now need to create the
>>>>>>>> appropriate nodes/edges on the graph.
>>>>>>>>
>>>>>>>> Currently trying to do this as a Generator, but running into some
>>>>>>>> problems.  Because this would run indefinitely in the background, the
>>>>>>>> controlling thread that updates the graph needs to be around whenever new
>>>>>>>> updates are available, but this doesn't seem to be the way that the
>>>>>>>> Generator class works.  Does it make more sense to do this as an Import?
>>>>>>>>
>>>>>>>> I'd appreciate any feedback you might have as I'm really close to
>>>>>>>> having this done!
>>>>>>>>
>>>>>>>> Thanks!
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> gephi-plugins mailing list
>>>>>>> gephi-plugins at lists.gephi.org
>>>>>>> http://gephi.org/mailman/listinfo/gephi-plugins
>>>>>>>
>>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> gephi-plugins mailing list
>>>>>> gephi-plugins at lists.gephi.org
>>>>>> http://gephi.org/mailman/listinfo/gephi-plugins
>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>> _______________________________________________
>>> gephi-plugins mailing list
>>> gephi-plugins at lists.gephi.org
>>> http://gephi.org/mailman/listinfo/gephi-plugins
>>>
>>>
>>
>>
>
> _______________________________________________
> gephi-plugins mailing list
> gephi-plugins at lists.gephi.org
> http://gephi.org/mailman/listinfo/gephi-plugins
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://gephi.org/pipermail/gephi-plugins/attachments/20101219/3aa53206/attachment-0001.html>


More information about the gephi-plugins mailing list