Meetup: Big Data on Google Cloud

We were invited to visit Google HQ in Bruxelles, Belgium. The building is very nice, easy to catch it by metro from Schuman, expect on the meetup day, there was a EU summit, everything was closed.

The topics were interesting. The presenters shared their experience with google Cloud services.

The Vente-Exclusive

Alex von Boxel presented their solution which uses google cloug services. The Vente-Exclusive is a webshop, they collect users data and behaviour, and try to create a customised offer at nigth and send your email.

Their concept is to collect 3TB data and load it to analyse automatically  with google cloud services.
If you check the shared presentation you will see they use cloud storage, dataproc, bigquery, dataflow from google. They use also not google solutionsd, the data is coming from a MS SQL databse, and they use tableau for reporting.

Google Cloud Platform at Vente-Exclusive.com from Alex Van Boxel

The BigQuery

We got a good show, explanation about Google BigQuery service from Matthias Feys (Datatonic).

Google BigQuery from Matthias Feys

DataFlow

Alex came back for a quick presentation about DataFlow. you could use Google Cloud SDK to reach the services. We know to use this services is also a question of money.  Their google bill is around 4.000 EURO per month!

Google Cloud Dataflow from Alex Van Boxel

How to mock new Date() calls in jUnit test

You can find lots of solutions for this.
I prefer to use Powermock, also because I use it for static code mocking.

The production code:

    @Override
    @Transactional
    public Integer submitTranslationRequest(RequestInfo requestInfo, DocumentTranslationRequestDTO requestDTO) throws PRDException {
         ...
        requestDTO.setSentPoetryDate(new Date());
        ....
    }

Let's see the TEST code:

@RunWith(PowerMockRunner.class)
@PrepareForTest(DocumentTranslationRequestServiceImpl.class)
public class DocumentTranslationRequestServiceImplTest {

    @InjectMocks
    private final DocumentTranslationRequestServiceImpl service = new DocumentTranslationRequestServiceImpl();

    @Test
    public void shouldSubmit() throws Exception {
        // given
        Date eventDate = new Date();
        PowerMockito.whenNew(Date.class).withNoArguments().thenReturn(eventDate);
        // when
        Integer result = service.submitTranslationRequest(requestInfo, dto);
        DocumentTranslationRequestDTO expectedDto = new DocumentTranslationRequestDTO();
        expectedDto.setSentPoetryDate(eventDate);
        // then
        verify(translationRequestDataProvider).saveRequest(eq(expectedDto));
        ...
  }
}

What we need:

        <dependency>
            <groupId>org.powermock</groupId>
            <artifactId>powermock-api-mockito</artifactId>
            <version>1.4.9</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>org.powermock</groupId>
            <artifactId>powermock-module-junit4</artifactId>
            <version>1.6.4</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>org.powermock</groupId>
            <artifactId>powermock-core</artifactId>
            <version>1.6.4</version>
            <scope>test</scope>
        </dependency>

If you have problem, error which complains for the javaassist, check the different versions, maybe there is a conflict.