[Gephi-plugins] Plugin for realtime graph generation based on web browsing

T B phreakocious at gmail.com
Sun Dec 19 19:51:49 CET 2010


More bug fixes, enhancements, etc.  Now you can specify another proxy to
forward requests through, and my branch includes a link to launch the plugin
and start a layout directly from the Welcome screen.  I believe this is just
about release ready.  I have a few questions for the developers, though.

1) When using the container method, is it possible to not have the
coordinates of nodes change when they are referenced repeatedly?  In
general, the node that represents the client gets a new edge on just about
every update.  Each time a new edge is added, the node is repositioned on
the graph to a new location.

2) On a similar note, is it possible to incrementally update the attributes
with new data?  It seems that each time a node is referenced, the attributes
are replaced, and their old values lost.  It would be nice to store that
state in the graph and not maintain separate data structures in the plugin.

Here are the links again:

Code available at: lp:~phreakocious/gephi/httpgraph
NBM at: http://phreakocious.net/dev/net-phreakocious-httpgraph.nbm



On Fri, Dec 10, 2010 at 10:21 AM, T B <phreakocious at gmail.com> wrote:

> Sure.. The largest nodes represent clients.  If you are the only one
> pointing at the proxy, you'll only have one client node.  The smallest nodes
> represent resources (aka URLs or URIs) (html/css, images, etc.)
>
> When your browser makes a request, a URI node is created for each resource,
> and an edge drawn from the client to that resource.  Also, if the request
> specifies a Referring page, a URI node is created for the referer and an
> edge drawn from the referer to that resource.  This allows the linking
> structure between pages to be revealed.
>
> Host nodes are created for the hosts that serve the resources, with the
> appropriate edges, and finally, domain nodes tie the hosts together.
>
> Each domain gets a color (up to 32) and the other nodes should get the
> domain color.
>
> NBM updated with some more bug fixes:
> http://phreakocious.net/dev/net-phreakocious-httpgraph.nbm
>
> Enjoiez!
>
>
> On Fri, Dec 10, 2010 at 3:23 AM, Mathieu Bastian <
> mathieu.bastian at gmail.com> wrote:
>
>> Cool I tested it and its working good :-)
>>
>> Some questions,
>> - could you explain what are the differents nodes, some of them have URLs,
>> some not
>> - what represent a link between two nodes
>> - does the node color mean something?
>>
>>
>> On Wed, Dec 8, 2010 at 8:25 PM, T B <phreakocious at gmail.com> wrote:
>>
>>> Switched from LittleProxy to Membrane:
>>> http://www.membrane-soa.org/soap-monitor/  Haven't had any issues..
>>>
>>> Launchpad updated, new NBM at
>>> http://phreakocious.net/dev/net-phreakocious-httpgraph.nbm
>>>
>>> Looking forward to hearing any feedback.
>>>
>>>
>>>
>>> On Tue, Dec 7, 2010 at 11:28 PM, T B <phreakocious at gmail.com> wrote:
>>>
>>>> Thanks, it has been stretching my Java abilities a bit.  I wasn't able
>>>> to rewrite it with Import as you suggest, but was able to get something
>>>> mostly functional from the advice.  This is still a Generator, but it stays
>>>> alive in the background updating the workspace until the app is closed.  All
>>>> you have to do is (install plugin), pick New Project,
>>>> File/Generate/HttpGraph, set your browser's proxy for 127.0.0.0:8088(or whatever port), and browse the web.
>>>>
>>>> Some issues:
>>>> Strange nodes with incomplete information
>>>> Nodes without edges??
>>>> Highly complex web sites (Google maps, for example) will bring
>>>> everything to a grinding halt
>>>>
>>>> Code available at: lp:~phreakocious/gephi/httpgraph
>>>> NBM at: http://phreakocious.net/dev/net-phreakocious-httpgraph.nbm
>>>>
>>>> Merci beaucoup!
>>>>
>>>>
>>>>
>>>> On Mon, Dec 6, 2010 at 11:43 PM, Mathieu Bastian <
>>>> mathieu.bastian at gmail.com> wrote:
>>>>
>>>>> That is cool stuff!
>>>>>
>>>>> Yeah Generator is not the best place for doing that. I would recommend
>>>>> to use Import directly, but woithout UI, to discard displaying the import
>>>>> report. As you were trying to do this with Generator you are probably
>>>>> already familiar with the container. Here is how to create, fill and append
>>>>> a container into the current workspace.
>>>>>
>>>>> //Get Currennt workspace
>>>>> ProjectController pc =
>>>>> Lookup.getDefault().lookup(ProjectController.class);
>>>>> Workspace workspace = pc.getCurrentWorkspace();
>>>>>
>>>>> //Get container
>>>>> Container container =
>>>>> Lookup.getDefault().lookup(ContainerFactory.class).newContainer();
>>>>>
>>>>> //Append container to current graph
>>>>> ImportController importController =
>>>>> Lookup.getDefault().lookup(ImportController.class);
>>>>> importController.process(container, new AppendProcessor(), workspace);
>>>>>
>>>>> Mathieu
>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Mon, Dec 6, 2010 at 1:29 PM, T B <phreakocious at gmail.com> wrote:
>>>>>>
>>>>>>> Gephi devs,
>>>>>>>
>>>>>>> The idea is to stream data about your web browsing directly into
>>>>>>> Gephi in real-time.
>>>>>>>
>>>>>>> After successfully proving out the concept with a Perl script and the
>>>>>>> Streaming plugin, I'm trying to implement it in Java.  I've embedded
>>>>>>> LittleProxy (http://www.littleshoot.org/littleproxy/) and extended a
>>>>>>> few classes to collect the necessary data and now need to create the
>>>>>>> appropriate nodes/edges on the graph.
>>>>>>>
>>>>>>> Currently trying to do this as a Generator, but running into some
>>>>>>> problems.  Because this would run indefinitely in the background, the
>>>>>>> controlling thread that updates the graph needs to be around whenever new
>>>>>>> updates are available, but this doesn't seem to be the way that the
>>>>>>> Generator class works.  Does it make more sense to do this as an Import?
>>>>>>>
>>>>>>> I'd appreciate any feedback you might have as I'm really close to
>>>>>>> having this done!
>>>>>>>
>>>>>>> Thanks!
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> gephi-plugins mailing list
>>>>>> gephi-plugins at lists.gephi.org
>>>>>> http://gephi.org/mailman/listinfo/gephi-plugins
>>>>>>
>>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> gephi-plugins mailing list
>>>>> gephi-plugins at lists.gephi.org
>>>>> http://gephi.org/mailman/listinfo/gephi-plugins
>>>>>
>>>>>
>>>>
>>>
>>
>> _______________________________________________
>> gephi-plugins mailing list
>> gephi-plugins at lists.gephi.org
>> http://gephi.org/mailman/listinfo/gephi-plugins
>>
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://gephi.org/pipermail/gephi-plugins/attachments/20101219/04376967/attachment.html>


More information about the gephi-plugins mailing list